Four Country Mobile Operator Study: Comparing UX Across Europe

Webinar
Four Country Mobile Operator Study:
a competitive benchmark of leading mobile operator sites

19th January 2011

Overview

!  Introductions

!  Study Background

!  What do we mean by UX Benchmarking? Why is it important?

!  Methodology and Technology

!  Findings and Opportunities

!  Questions and Comments

!  UserZoom and UX Alliance Partners at a glance

Introductions

Arthur Moan Louiselle Morand
Country Manager UK & Ireland User Experience Consultant
at UserZoom! at Telono!

Martje van der Linde Javier Darriba
User Experience Consultant Managing Director
at User Intelligence! !
at Xperience Consulting

Today’s Session Goals

!  See how users in four countries look for:
!  the monthly cost to surf the internet daily with an iPad

!  the fax number/address for a high street shop

!  To gain insight into the European mobile operator market

!  Introduction and review of UX Benchmarking methodology

!  Introduction and review of UserZoom software solution for
conducting this type of research

Study Background

!  "#$%&''(&)*&+,-$./0)#1)2&.34+5)$#*42&)#%&/.6#/)
9#.27) 746&78)

:&*746&7)
.+.2;<&3)

! =./',4%.+67)7&2&,6&3)1/#$)1#>/),#>+6/4&7)?.%%/#@)ABBCD)
=.+&2) E&6-&/2.+37F)G%.4+F)GH46<&/2.+3).+3)6-&)IJ)

! K4+3)6-&)$#+6-2;),#76)6#)7>/1)6-&)4+6&/+&6)3.42;)H46-).+)4=.3)
O.707) ! K4+3))6-&)1.@)+>$*&/)L)6-&).33/&77M)#1).)7-#%)+&./).)*45),46;)
6/.4+)76.'#+8)))?MN&%&+34+5)#+)6-&),#>+6/;C)

What is UX benchmarking and why is it
important?

OK……. So how do we define UX benchmarking?

important?

!  Definition of Benchmarking:
!  The process of comparing the business
processes and performance metrics
including cost, cycle time, productivity,
or quality to another that is widely
considered to be an industry standard
benchmark or best practice.
!  It provides a snapshot of the
performance of your business and helps
you understand where you are in relation
to a particular standard.

important?
!  Definition of UX:
!  It’s about how a person feels about using
a system. It highlights the experiential,
affective, meaningful and valuable
aspects of HCI and product ownership,
but it also covers a person’s perceptions
of the practical aspects such as utility,
ease of use and efficiency of the system.
!  User experience is subjective in nature,
as it is about an individual’s feelings and
thoughts about the system. User
experience is dynamic, because it
changes over time as the circumstances
change.

important?

Some may wonder: is UX benchmarking:

?

Not really…..
Here’s how we’d define UX Benchmarking:

•  The process of comparing a website’s
performance, specifically in relation to UX
metrics, including among others effectiveness,
efficiency, overall satisfaction and quality
perception (ISO-9241-11 standard), to another
that is widely considered to be an industry
standard benchmark or best practice.

•  UX benchmarking provides a snapshot of the
performance of your website and helps you
understand where you are in relation to a
particular standard.

10 must-dos with UX Benchmarking

1.  Define measurable & comparable UX-related KPIs (key performance indicators)

2.  Select competitors to benchmark against

3.  Define target audience

4.  Identify common site activities/tasks (between-subjects design)

5.  Conduct research using scenario-based testing

6.  Gather both quantitative & qualitative data

7.  Use the quantitative data to test for statistically significant differences between
means or proportions

8.  Use qualitative data to support your findings and understand the why!

9.  Research at least twice a year to measure design changes and trends in
expectations

10. Use the right research solution/tool

important?

‘You can’t manage what you can’t measure’
Attributed to Peter Drucker, Mgt Guru

The question is:

How are you actually measuring UX and defining
your benchmark?

Common UX research challenges

!  Difficulties obtaining objective, statistically significant, actionable data

!  Obtaining idea adoption from stakeholders

!  Geographic/demographic limitations

!  Research is costly and time-consuming

How does UserZoom add value?

!  By allowing you to see the
BIG picture
Behaviour

Usability Lab Web Traffic
Studies Analysis
!  Testing large samples of
geographically spread
participants
Online
UX
Studies !  Combining statistically-
significant usability + user
behaviour data
Attitudes

Online !  Measuring Customer
Focus Groups Surveys Experience cost-effectively

Qualitative Quantitative !  Getting actionable insights

Online Usability Testing Solution

!  UZ Self-Serve Edition
!  100% on-demand, web-
based software solution
!  To manage & conduct
various types of online UX
research projects

Our method

•  Hundreds of users participate in a study
•  In their natural context
•  From geographically spread locations
•  Users try to complete tasks (or goals) + answer questions
•  No human moderation needed
•  Our browser bar connects users with our secure servers

Analyze data real-time using UZ Analytics

Effectiveness + efficiency ratios

Analyze data using UZ Analytics

Qualitative feedback collected. Easy visualization through ‘text clouds’

Analyze data using UZ Analytics

UserZoom Results - Behavioural

Study structure
Introduction:
!  For each study, participants were randomly assigned two tasks on two popular
mobile operator websites (four tasks in total)
!  Task: Find a monthly iPad data plan
!  Task: Locate a store address/fax number

Post-task:
Mobile operator site evaluation ratings included -
!  Ease of use
!  Perceived complexity of sites
!  Perceived consistency of sites
!  Clear where to start
!  Confidence in using the sites
!  Usability Satisfaction (SUS score)
!  What was liked most about the sites
!  What was liked least about the sites

Wrap-up:
!  Mobile Operator Site Preference
!  Likelihood to recommend: (Net Promoter Score)

Switzerland Study – Participant Demographics

=./',4%.+67D) UBBR)
EW)XQ)%./',4%.+67)

QBR)
N&$#5/.%-4,)3.6.),#22&,6&3D)
• )9&+3&/)
• )">//&+6)$#*42&)#%&/.6#/) TBR) PQR)
• )Y5&)
• )Z+34(43>.27)*&6H&&+)UQ[PP);&./7)#1)
.5&)H&/&)4+(46&3)6#),#$%2&6&)6-&) ASR)
76>3;8) ABR)

SBR)
9EN])
AP)1&$.2&7),#$%2&6&3)6-&)76>3;)
^^)$.2&7),#$%2&6&3)6-&)76>3;)
BR)
K&$.2&) V.2&)

Switzerland Study – Participant Demographics

XBR)
V_aZb)_=]YO_]) P`R)
TBR)
GH477,#$F)G>+/47&).+3)_/.+5&)
H&/&)6-&)$#76),#$$#+) PBR)

,>//&+6)%/#(43&/7).$#+5) ABR)
%./',4%.+678) ^BR)
• )c_6-&/dD));.22#F)(6@F)e#3.1#+&) UQR)
SBR) UPR)
UBR) ^R) AR)
BR) UR)
BR)
PBR) GH477,#$) _/.+5&) G>+/47&) O&2&S) V[a>35&6) "##%) _6-&/)
V#*42&) V#*42&)

ABR)
^SR)
Y9)
^BR) STR)
SAR) O-&)-45-&76)%&/,&+6.5&)#1)
SBR) UQR) GH477)%./',4%.+67)./&))))))))))
^T)[)AP);&./7)#1).5&8)
UBR) • )UA)%./',4%.+67)./&)UQ[SP)
• )U`)%./',4%.+67)./&)ST[^P)
BR) BR)
BR) • )SP)%./',4%.+67)./&)^T[AP)
I+3&/)UQ) UQ[SP) ST[^P) ^T[AP) AT[PP) _(&/)PP) • )SB)%./',4%.+67)./&)AT[PP)

Netherlands Study – Participant Demographics

=./',4%.+67D) UBBR)
EW)UBB)%./',4%.+67)

QBR)
N&$#5/.%-4,)3.6.),#22&,6&3D)
• )9&+3&/)
• )">//&+6)$#*42&)#%&/.6#/) TBR) P^R)
• )Y5&) AXR)
• )Z+34(43>.27)*&6H&&+)UQ[PP);&./7)#1)
.5&)H&/&)4+(46&3)6#),#$%2&6&)6-&) ABR)
76>3;8)

SBR)
9EN])
AX)1&$.2&7),#$%2&6&3)6-&)76>3;)
P^)$.2&7),#$%2&6&3)6-&)76>3;) BR)
K&$.2&) V.2&)

Netherlands Study – Participant Demographics
PBR)

V_aZb)_=]YO_])
ABR)
e#3.1#+&F)J=E).+3)O[V#*42&)
^BR)
H&/&)6-&)$#76),#$$#+) ^BR)
SAR)
,>//&+6)%/#(43&/7).$#+5) SUR)
%./',4%.+678) SBR)
UPR)
• c_6-&/dD))-4F)74$;#F)2&*./.F)
<455#F)/.*#)$#*4&2) UBR) XR)
^R)
BR)
J=E) e#3.1#+&) O&21#/6) O[V#*42&) a&+) _6-&/)
PBR)
AUR)
ABR)

Y9)
^BR)
S^R) O-&)-45-&76)%&/,&+6.5&)#1)
SUR)
SBR) N>6,-)%./',4%.+67)./&))))))))
UPR)
^T)[)AP);&./7)#1).5&8)
UBR) • )SU)%./',4%.+67)./&)UQ[SP)
• )UP)%./',4%.+67)./&)ST[^P)
BR) BR)
BR) • )AU)%./',4%.+67)./&)^T[AP)
I+3&/)UQ) UQ[SP) ST[^P) ^T[AP) AT[PP) _(&/)PP)
• )S^)%./',4%.+67)./&)AT[PP)

Spain Study – Participant Demographics

=./',4%.+67D) UBBR)
EW)UBB)%./',4%.+67)

QBR)
N&$#5/.%-4,)3.6.),#22&,6&3D)
• )9&+3&/)
• )">//&+6)$#*42&)#%&/.6#/) TBR) PPR)
• )Y5&) APR)
• )Z+34(43>.27)*&6H&&+)UQ[PP);&./7)#1)
.5&)H&/&)4+(46&3)6#),#$%2&6&)6-&) ABR)
76>3;8)

SBR)
9EN])
AP)1&$.2&7),#$%2&6&3)6-&)76>3;)
PP)$.2&7),#$%2&6&3)6-&)76>3;) BR)
K&$.2&) V.2&)

Spain Study – Participant Demographics

PBR)
V_aZb)_=]YO_])
V#(476./F)e#3.1#+&).+3) ABR)
^AR)
^XR)

_/.+5&)H&/&)6-&)$#76)
^BR)
,#$$#+),>//&+6)%/#(43&/7) S^R)
.$#+5)%./',4%.+678) SBR)

UBR)
SR) ^R)
UR)
BR)
PBR) _/.+5&) e#3.1#+&) V#(476./) f#45#) G4$;#) _6-&/)
A^R)
ABR)
Y9)
^BR) SXR) O-&)-45-&76)%&/,&+6.5&)#1)
SSR)
SBR)
G%.+47-)%./',4%.+67)./&)))
ST[^P);&./7)#1).5&8)
UBR) QR) • )SX)%./',4%.+67)./&)UQ[SP)
• )A^)%./',4%.+67)./&)ST[^P)
BR) BR)
BR) • )SS)%./',4%.+67)./&)^T[AP)
I+3&/)UQ) UQ[SP) ST[^P) ^T[AP) AT[PP) _(&/)PP) • )BQ)%./',4%.+67)./&)AT[PP)

United Kingdom Study – Participant Demographics

=./',4%.+67D) UBBR)
EW)UBU)%./',4%.+67)

QBR)
N&$#5/.%-4,)3.6.),#22&,6&3D)
TPR)
• )9&+3&/)
• )">//&+6)$#*42&)#%&/.6#/) TBR)
• )Y5&)
• )Z+34(43>.27)*&6H&&+)UQ[PP);&./7)#1)
.5&)H&/&)4+(46&3)6#),#$%2&6&)6-&) ABR) ^PR)
76>3;8)

9EN]) SBR)
TP)1&$.2&7),#$%2&6&3)6-&)76>3;)
^P)$.2&7),#$%2&6&3)6-&)76>3;)
BR)
K&$.2&) V.2&)

United Kingdom Study – Participant Demographics

V_aZb)_=]YO_]) PBR)

e#3.1#+&F)_S).+3)_/.+5&) ABR)
H&/&)6-&)$#76),#$$#+)
,>//&+6)%/#(43&/7).$#+5) ^BR)
SAR) S^R)
%./',4%.+678) UXR)
SBR)
• )c_6-&/dD))Y73.F)O&7,#F)e4/54+) USR)
UAR)
UBR)
UBR)

BR)
_/.+5&) e#3.1#+&) _S) O[V#*42&) O-/&&) _6-&/)

PBR)

ABR) Y9)
^BR) ^UR)
^BR) O-&)-45-&76)%&/,&+6.5&)#1)
SUR) IJ)%./',4%.+67)./&))^T)[)AP)
UQR)
SBR) ;&./7)#1).5&8)
• )SU)%./',4%.+67)./&)UQ[SP)
UBR) • )^B)%./',4%.+67)./&)ST[^P)
BR) BR) • )^U)%./',4%.+67)./&)^T[AP)
BR) • )UQ)%./',4%.+67)./&)AT[PP)
I+3&/)UQ) UQ[SP) ST[^P) ^T[AP) AT[PP) _(&/)PP)

Executive Summary

!  Orange UK and Orange Switzerland (CH) had highest task success
rates for both iPad (UK 75%) and Address (CH 87%) tasks.
!  Orange Spain (SP) had the lowest success rate across tasks 24%,
the lowest participants satisfaction scores, and substantially
lower preference (25%) vs. Vodafone (75%).
!  All mobile providers tested in each country received low usability
satisfaction scores by participants.
!  The lowest system usability scale scores (SUS) were received by
Vodafone Spain (51) and Orange Spain (40).
!  All four mobile providers received low Net Promoter Scores
(between -46% and -75%). There were more detractors than
promoters in every instance.

Top level overview of Task 1 across all
countries with deep dive into Switzerland &
UK results

“Find the monthly cost to surf the internet
daily with an iPad”

O.70)V&6/4,7D)4=.3))V#+6-2;)=2.+)

1+02'-.//&00'' !"#$%&'()&*+,"*' -.//&00)
_/.+5&?IJC)
_/.+5&?G=C) U) )_/.+5&)?IJC) XPR)
_/.+5&?"gC)
S) e#3.1#+&)?G=C) TUR)

e#3.1#+&?G=C) ^) e#3.1#+&)?IJC) PBR)
e#3.1#+&?IJC)
A) O[V#*42&)?EbC) ABR)
e#3.1#+&?EbC)
P) GH477,#$)?"gC) ^BR)
O[V#*42&?EbC)
T) _/.+5&)?G=C) SQR)

GH477,#$?"gC) X) e#3.1#+&)?EbC) SXR)

BR) SBR) ABR) TBR) QBR) UBBR) Q) _/.+5&)?"gC) SAR)

Switzerland: Swisscom
!  Successful process
!  “Internet”
!  “Internet with your Notebook”
!  “iPad tariffs”
!  Can be found in 3 clicks if
browsing

Heatmaps First Click: Swisscom - Switzerland (DE)
O.70D))4=.3))V#+6-2;)=2.+)

Overall 30% were successful
-  6% used the search bar
- 40% rolled over to the side navigation bar and then clicked on the links on the right
-  15% clicked “Kombi-Angebote” on the side navigation bar

Heatmaps First Click: Orange – Switzerland (DE)
O.70D))4=.3))V#+6-2;)=2.+)

Overall 24% were successful
-  14% used the search bar
-  16% clicked “Mobile” on the top navigation bar
-  14% clicked “Internet” on the top navigation bar

Heatmaps First Click: Orange - Spain
O.70D))4=.3))V#+6-2;)=2.+)

_(&/.22)SQR)H&/&)7>,,&771>2)
[ )SUR)>7&3)6-&)7&./,-)*./)
[ )UAR),24,0&3)c$#(42d)#+)6-&)6#%)+.(45.'#+)*./)
[ )TPR),24,0&3)4+)#6-&/)%2.,&7)

Heatmaps First Click: Vodafone - Spain
O.70D))4=.3))V#+6-2;)=2.+)

_(&/.22)TUR)H&/&)7>,,&771>2)
[ )XR)>7&3)6-&)7&./,-)*./)
[ )^UR),24,0&3)cO4&+3.)_+24+&d)#+)6-&)6#%)+.(45.'#+)*./)
[ )UAR),24,0&3)4+)6-&),&+6&/)%/#$#'#+.2)./&.)

Heatmaps First Click: Vodafone - Netherlands
O.70D))4=.3))V#+6-2;)=2.+)

_(&/.22)SXR)H&/&)7>,,&771>2)
[ )AR)>7&3)6-&)7&./,-)*./)
[ )UAR),24,0&3)cZ+6&/+&6d)#+)6-&)6#%)+.(45.'#+)*./)
[ )U`R),24,0&3)4+)6-&),&+6&/)%/#$#'#+.2)./&.)

Heatmaps First Click: T-Mobile - Netherlands
O.70D))4=.3))V#+6-2;)=2.+)

_(&/.22)ABR)H&/&)7>,,&771>2)
[ )UPR)>7&3)6-&)7&./,-)*./)
[ )U^R),24,0&3)cY*#++&$&+6d)#+)6-&)6#%)+.(45.'#+)*./)
[ )SXR),24,0&3)cZ+6&/+&6d)#+)6-&)6#%)+.(45.'#+)*./)

Heatmaps First Click: Orange - United Kingdom
O.70D))4=.3))V#+6-2;)=2.+)

_(&/.22)XPR)H&/&)7>,,&771>2)
[ )U^R)>7&3)6-&)7&./,-)*./)
[ )^XR),24,0&3)c4=.3d)#+)6-&)2&h)+.(45.'#+)*./)
[ )PBR),24,0&3)4+)#6-&/)%2.,&7)

Vodafone > iPad: UK

•  Only half of the participants were able to successfully find the
monthly cost of an iPad subscription on the Vodafone website.

•  Successful Participants spent an average of 5 minutes 24 seconds and
19 clicks completing the task.

Task Effectiveness

Orange > iPad: UK

•  The majority of participants were able to find the correct price for
the plan (75%).

•  Successful Participants spent an average of 1 minute 11 seconds and
7 clicks completing the task.

Task Effectiveness

Top level overview of Task 2 across all
countries with deep dive into The
Netherlands & Spain’s results

“Locate a store address/fax number”

O.70)V&6/4,7D)b#,.'+5)G6#/&)Y33/&77L=-#+&)+>$*&/))

1+02'-.//&00'
!"#$%&'()&*+,"*' -.//&00)
_/.+5&)?"gC))
_/.+5&)?IJC))
U) _/.+5&)?"gC)) QXR)
_/.+5&)?G=C))
S) O[V#*42&)?EbC)) XXR)
e#3.1#+&)?G=C))
^) GH477,#$)?"gC)) XTR)
e#3.1#+&)?EbC))
e#3.1#+&)?IJC)) A) _/.+5&)?IJC)) XUR)

P) e#3.1#+&)?G=C)) TQR)
O[V#*42&)?EbC))
T) _/.+5&)?G=C) P^R)
GH477,#$)?"gC))
X) e#3.1#+&)?EbC) PSR)
BR) SBR) ABR) TBR) QBR) UBBR)
Q) e#3.1#+&)?IJC) A`R)

Success rates for Netherlands

!  Task 2 : «!Find the closest shop to the Amsterdam railway station
and write down its address.!»
!  The T-mobile site provided the best results regarding this task,
77% of the participants was successful.
!  On average it took slightly over one minute to find the right
information, with an average of only 4 page views and 8 clicks.

!  On the Vodafone website 52% was successful in achieving this
information.
!  Remarkable result is that it took an average of 3 minutes and 29
clicks to complete this task.

Success questionnaire - Netherlands

!  Nevertheless, the rate of success regarding a task is not
necessarily an indication of the ease of use:
!  The Vodafone website was less successful, but only 28% of the
participants regarded it to be difficult to acquire the needed
information.

!  On the T-mobile website, people were far more successful, but also
less satisfied with the level of difficulty to get to the right URL: 42%
of the participants thought this was (very) hard to achieve.

!  Vodafone

!  T-mobile

Abandon questionnaire - Netherlands

!  Main reasons for abandoning a task were:
!  Page not loading: Especially the Vodafone website was unable to load
the required page. Remarkable is that people wait up to 5 minutes!

!  Information not in expected location: Both Vodafone and T-mobile
present information in an unexpected manner. Participants have
certain expectations on where and how to find the required
information. In case these expectations are not met, people easily
abandon the website.

QUOTES
“I was looking for it under Customer service and expected to be able to fill in my
location and that it would automatically give the closest store location.”
“ I was unable to open the page.”

Final questionnaire - Netherlands

!  Users of both the Vodafone as well as the T-mobile website are
neither positive nor negative about the websites:
!  They regard it as easy to use for inexperienced users and there is no
need for extra help.

!  Nevertheless, the lack of success will have some influence on the
use of the website.
!  Most users indicate that they will not likely be using these websites in
the near future again.

Task 2 Spain : Find the closest shop to the Atocha

railway station and take note of its fax number. ! )

Task 2 Spain – success ratios

•  On Vodafone 68% of users •  On Orange 53% of users completed
completed the task successfully . the task successfully.

Vodafone analysis - Spain
•  First click of users was mostly focused on the search engine
and the help section. There were several users that tried to
find the shops using the Vodafone logo.


•  The reason users got the answer
incorrect was due to a problem
on the web. Although as
happened in the previous task,
the information that users found
was not clear enough.

The section with the The site is not clear nor Too much information on
Vodafone shops is easy, intuitive, you have to use the map
the problem is one you the search engine, the
have to find one in Madrid information provided is not
precise

•  The reasons participants
abandoned the task was
mainly due to the
information being
incomplete (67%). Other
reasons were:
•  The information
presentation is not clear
enough
•  The information,
options and menus are I have entered province
difficult to understand “Madrid”, city “Madrid” and
no results appeared with
•  The information wasn’t this search options. I have
in the expected location tried with different options
and it always happened
•  Few search options
the same.
•  As it has been seen in the
previous task users again had
problems with the
information provided.

Orange Analysis - Spain )
•  Most of the users went to the tab “Orange shop” in order to look
for the addresses of the physical shops.

Orange Analysis - Spain

•  The reason for the incorrect
answer was ‘not being able to
find the correct information
due to options and menus
being difficult to understand.’
•  Users explained that the site
mainly focuses on the iPhone
whereas iPad information is
quite difficult to find.
•  The system could be improved
There is no way to
using an accessible and easy understand what is going
to find search engine. on here.

There is no way to find
anything. What a fuss!

Orange Analysis - Spain

•  Those users who didn’t finish
the task was mainly because
they couldn’t find the
information where they were
expecting to find it.
•  The links to the information
were not clear enough.
•  The site is very focused on
selling products or services
instead of giving clients the
I don’t have anything
service that they need. against Orange but its
website is a disaster

K4+.2)i>&7'#++.4/&)j).22),#>+6/4&7D))
:-4,-)$#*42&)#%&/.6#/)3#);#>)%/&1&/k)

E&6-&/2.+37) GH46<&/2.+3)

e#3.1#+&) _/.+5&)
AXR) ATR)
O[V#*42&) GH477,#$)
P^R) PAR)

G%.4+) IJ)

_/.+5&)
SPR)

e#3.1#+&)
ATR)
_/.+5&)
PAR)
e#3.1#+&)
XPR)

Metrics: SUS Score
The System Usability Scale (SUS) is a set of 10 questions that is used to
measure the perception of a systems overall usability. Participants rate each
question with a 1 to 5 scale (5 being strongly agree).

1. I think that I would like to use this system frequently.
2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I think that I would need the support of a technical person to be able to use this
system.
5. I found the various functions in this system were well integrated.
6. I thought there was too much inconsistency in this system.
7. I would imagine that most people would learn to use this system very quickly.
8. I found the system very cumbersome to use.
9. I felt very confident using the system.
10. I needed to learn a lot of things before I could get going with this system.

V&6/4,7D)GIG)G,#/&)
Yh&/),#$%2&'+5)6-&)6.707)#+)&.,-)$#*42&)%/#(43&/)H&*746&)%./',4%.+67)H&/&).70&3)6#)
,#$%2&6&)6-&)GIG)l>&7'#++.4/&8)O-&).(&/.5&7)1#/)&.,-)l>&7'#+)./&)>7&3)6#),.2,>2.6&)
.+)#(&/.22)7,#/&)?B[UBBC8))

34) -)+$5) 6&,7&*%+580) -9$,:&*%+58)

e#3.1#+&) PS) PU) PT) ELY)

_/.+5&)) PP) AB) ELY) P`)

GH477,#$) ELY) ELY) ELY) PA)

O[V#*42&) ELY) ELY) PP) ELY)

GIG)G,#/&)9>43&MD)^PW%##/F)PBW_JF)XBW5##3F)QPW&@,&22&+6)

a.+5#/F)Y8)?SBB`F)V.;C8)N&6&/$4+4+5):-.6)Z+34(43>.2)GIG)G,#/&7)V&.+D)Y334+5).+3)Y3m&,'(&)].'+5)G,.2&8)n#>/+.2)#1)
I7.*4246;)G6>34&7F)A?^CF)UUA[US^8))

E&6)=/#$#6&/)G,#/&)?E=GoC)
After completing the four tasks, participants were asked how likely they
would be to recommend each mobile provider’s website.
0 1 2 3 4 5 6 7 8 9 10 NPS

Orange (SP) 16% 11% 15% 10% 7% 14% 9% 9% 2% 4% 3% -75%

Orange (UK) (n=101) 10% 3% 4% 7% 7% 21% 11% 10% 14% 9% 4% -50%

Orange (CH) (n=78)
9% 4% 12% 4% 10% 23% 9% 15% 12% 0% 3% -68%
Swisscom (CH) (n=78)
-51%
4% 1% 15% 6% 5% 15% 12% 22% 12% 3% 5%
T-Mobile (NL) 13% 4% 6% 4% 6% 27% 13% 10% 13% 3% 1% -69%

Vodafone (NL) 8% 3% 9% 8% 5% 22% 18% 11% 9% 3% 4% -66%

Vodafone (SP) 7% 1% 4% 6% 14% 13% 13% 18% 12% 8% 4% -46%

Vodafone (UK) 8% 1% 4% 8% 9% 22% 13% 16% 9% 5% 5% -55%

Detractors Promoters

6;-' W)
<'"=';*">",&*0'
?@0'+58'AB0C' [) D&,*+/,"*0'
?B',7*".E7'FC'

Top 5 findings / Learnings
!  The satisfaction is directly related with the success of the users in
obtaining the information that they are looking for. In general the
websites for mobile operators in all countries are not performing optimal
when consumers have a task in mind. The performance rate was
dependent on the task, not the website.

!  Across the board the main difficulties were in participants finding
specific information. The normal way users tried to find the information
was through the menu structure. When they realized it was too
complicated they tried to use ‘search’ although the results weren’t
always clear enough.! A much clearer architecture and clearer results in
search engines are therefore needed. Furthermore the use of
promotional ads made the navigation much longer and difficult for most
of the users.


!  Analysing NPS, SUS, success on task and preference data it is clear that
the brand halo effect has a bearing on site preference in each market
irrespective of task success.

!  UK - although participants had notably higher task success rates on
the Orange site (75% and 71%) vs. the Vodafone site (50% and 49%),
when asked for site preference at the end of the study, the Orange
website was selected only 8% more than the Vodafone site (54% vs
46%) respectively. Out of the Orange customers (n=17) more than half
(9) preferred the Vodafone website.

!  Orange is not strong in Spain

!  Swisscom the national provider is stronger in Switzerland

!  Vodafone marginally outperforms T-mobile in the Netherlands

!  When participants were asked whether they would recommend the sites
in each market all operators scored negatively showing room for
improvement. Although these scores are low, previous research has shown
that telecommunication companies have the lowest net promoter scores
within the technology sector.

!  As Vodafone and Orange were both tested in 3 of the 4 markets we can
benchmark their experiences.

!  Vodafone outperforms Orange on aggregated SUS (53 versus 51.3)

!  Vodafone outperforms Orange on aggregated NPS (-55% versus -64.3%)

!  Task 1 success aggregated : Vodafone outperforms Orange (46% versus
42%)

!  Task 2 success aggregated : Orange outperforms Vodafone (70% versus
56%)

Information on UI

!  User Intelligence is a user experience consultancy and
was founded in 2002

!  Offices in Amsterdam and Helsinki, additional service offering in
Sweden and Norway

!  Our team consists of 17 consultants with a passion for people and
interactive technology

About Telono

!  Telono is a Swiss consultancy
specialising in usability, user
experience and web
accessibility.

!  We help companies implement
a systematic user-centred
design process, from user
experience strategy and user
research to interaction design
and usability evaluations.

!  Telono is the Swiss
representative of the User
Experience Alliance
(UXalliance)

About Xperience Consulting

Usability and user experience experts, we
create interfaces that suit users’ needs and
objectives of your company. Intuitive and easy
to use websites. •  10 years of experience

•  2 offices (Barcelona & Madrid)

•  150 AI projects

•  300 usability studies

•  6000 users analyzed

•  150 clients (15 countries)

UX Alliance - Who are we?

!  The UX Alliance is an international network for usability testing and user experience
research consisting of core partners, associates, and affiliate members.

!  Back in 2005 the leading user experience companies from France, Germany, Spain, UK
and US met in Barcelona with a single common goal in mind: how to provide effective
international user experience services to our clients.

!  Having worked together on international
projects for many years, we recognised
that we were ideally placed to lead
international user experience research.
Thus, the UXalliance was born…

www.uxalliance.com

Who are we?
!  SB)"#>+6/4&7)j)SS)"#$%.+4&7)")_(&/)UBB)52#*.2)%/#m&,67)%&/);&./)6#5&6-&/8)

SirValUse User Intelligence ExperienceLab UIDesign Group Think User
Germany Netherlands/ United Kingdom Russia Korea
Finland
Axance
France User Experience
China
ExperienceU
Czech Republic
Yu Centrik
Canada
Snitker & Co.
Denmark
Ayer Viernes
Xperience Consulting Chile
Spain

Telono Symetria
Switzerland UXa Partners, Associates, & Affiliates Poland

Optimal Experience Mercedes Sanchez User Centric, Inc. Mitsue Links Optimal Usability
Australia Brazil United States Japan New Zealand

www.uxalliance.

About UserZoom

!  Leading online (remote) user
experience research software
company

!  9 years experience in UX
research & consulting

!  Offices in Sunnyvale (CA),
London (UK) and Barcelona
(Spain)

!  Created because traditional UX
research methods are great but
not enough

Thanks so much for your time!

Check out our upcoming events at www.userzoom.com

Sunnyvale (USA) Barcelona (Spain) Cheshire (UK)

440 N. Wolfe Rd. Sunnyvale, Av. Diagonal 419 3º 2ª 08008 Booths Park. Chelford Road, Knutsford,
CA 94085 Barcelona Phone: +34 93 414 Cheshire WA16 8GS
Phone: +1 (408) 524 7445 7554 Fax: +34 93 209 8380 Phone: + 44 (0)1565 759890
Contact: Alfonso de la Nuez Contact: Xavier Mestres Cell: + 44 (0)7900 472 920
Contact: Arthur Moan

Four Country Mobile Operator Study: Comparing UX Across Europe

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie Four Country Mobile Operator Study: Comparing UX Across Europe

Ähnlich wie Four Country Mobile Operator Study: Comparing UX Across Europe (20)

Mehr von UserZoom

Mehr von UserZoom (20)

Four Country Mobile Operator Study: Comparing UX Across Europe