...

1. Mobile Banking
Benchmark Study
Authors: Ann Rochanayon, Sneha Kanneganti
www.userzoom.com

2. Study Methodology
MOBILE BANKING BENCHMARK STUDY
UserZoom’s researchers created a competitive benchmark
study to determine which bank offers the best mobile
experience for prospective customers and to assess the impact
of the mobile banking experience on brand perception, Net
Promoter Score and conversion.
200 panelists were recruited by email from SSI, Survey
Sampling International during the February 24 – March 2014
timeframe. The participants met the following criteria:
US participants were randomly assigned to evaluate one of the
three US Bank sites, while Canadian participants evaluated
CIBC. 50 panelists evaluated each site.
Study Methodology
Sites evaluated
Participants
• Ages 26+
• Not a current customer of the site evaluated
• Have prior experience with mobile banking
• Participated in the study via smartphone
5

3. 7
Finding an ATM / branch and researching banking services (two of the three tasks performed) are among top banking
activities performed via smartphone.
Selected “Smartphone” when asked “On which of devices, have you conducted the following online banking activities listed
below?”
Banking Activities Conducted via Smartphone
Banking Activities Conducted via Smartphone
MOBILE BANKING BENCHMARK STUDY
Banking Activities Conducted via Smartphone
Check Account Balance
View Transactions
Find an ATM / Branch
View / Pay Bills
View Account Statement
Transfer Money
Research Banking Services
(e.g., accounts, credit cards, loans)
Deposit Money
90%
87%
76%
74%
72%
71%
60%
43% N=198

4. 9
Participants found it easiest to research accounts on CIBC compared to the other sites and
encountered the least number of issues when performing the three tasks.
Key Findings
MOBILE BANKING BENCHMARK STUDY
Although task success was very low for researching an account, ease of use ratings and
frustration ratings were not significantly different from the top site (CIBC).
Participants found KeyBank easiest to use when finding customer service information and also
rated KeyBank highest when finding a branch.
KeyBank tied with CIBC for the highest overall satisfaction ratings
and the highest Net Promoter Score.
Participants found the site harder to use when researching accounts and finding customer
service information and generally found that too many clicks were required to find information.
USAA performed third-best in the study.
CIBC had the highest overall ratings.

5. 10
Key Findings
MOBILE BANKING BENCHMARK STUDY
Although Wells Fargo has a mobile optimized website, participants were forced to navigate to
the full site to research an account.
For task #2, locating a customer service phone number, 22% reported frustration in using the
full site, even though this information was available on the mobile site.
Clicking on the incorrect link was among the top problems.
Wells Fargo had the lowest overall ratings for ease of use,
organization and layout and satisfaction primarily due to difficulties
in navigation and participants having to use the full site to perform
the tasks
Participants brand perception improved significantly
from pre to post mobile site evaluation
Net Promoter Score for Wells Fargo was lowest overall (-44%), while KeyBank had
the highest Net Promoter Score (8%)
Wells Fargo had the smallest improvement in brand perception, while the
brand perception improvements on other sites were comparable.
Bank
Wells Fargo
KeyBank
USAA
CIBC
Delta
+13%
+31%
+32%
+27%

6. 12
Overall Results
MOBILE BANKING BENCHMARK STUDY
Key Performance Metrics
After study participants completed the three tasks on one of the banking sites, they answered questions about their
overall site experience and questions about brand perception.
CIBC rated highest overall on key performance metrics. Meanwhile, Wells Fargo rated lowest on most metrics and
significantly lower than CIBC on ease of use. KeyBank tied CIBC for overall satisfaction and had the second highest overall
ratings, followed by USAA.
Ease of use, Organization and Layout, and Satisfaction: Responded 6-7 on a 7-point scale when asked, “Please rate your experience with the following
aspects of the mobile site.” where 1 = Negative and 7 = Positive
Likelihood to open an account: Responded 6-7 on a 7-point scale when asked, “Based only on your experience today, how likely would you be to open
a bank account with [BANK]?” where 1 = Not at all likely and 7 = Extremely likely
Key Performance Metrics
Ease of Use
Organization and Layout
Satisfaction
Likelihood to open an account
30%
36%
34%
16%
56%
46%
50%
16%
46%
40%
42%
14%
58%
50%
50%
22%
*Significantly lower than site with highest rating at 90% Cl
*
Site with the highest rating Site with the lowest rating

7. 15
Overall Results
MOBILE BANKING BENCHMARK STUDY
Study participants were asked how
they perceived the brand they
evaluated prior to interacting with
the site and also after interacting
with the site. All sites improved in
brand perception after the site
interaction, however, Wells Fargo
experienced the smallest increase in
brand perception.
Based on what you know / your experience today, what is your perception of [BANK]?
Responses of 6-7 shown from a 7-point scale where 1 = Negative and 7 = Positive
13%
11% 12%
17%
42% 44% 44%
26%
N=48 N=50 N=18 N=50 N=25 N=50 N=46 N=50
Pre Brand Perception Post Brand Perception
Pre vs. Post Brand Perception
Bank
Wells Fargo
KeyBank
USAA
CIBC
Delta
+13%
+31%
+32%
+27%

8. 16
Overall Results
MOBILE BANKING BENCHMARK STUDY
Change in Associated Brand Attributes
Pre vs. Post Mobile Site Evaluation
Study participants also selected brand attributes they associated with the site pre and post site experience. The chart
below shows the change in positive brand association. Many positive brand attributes increased for many of the sites,
although there are several decreased for Wells Fargo and KeyBank.
Convenient
Competitive
Comprehensive
Reliable
Trustworthy
Secure
Easy access
Exclusive
+Positive
Which of the following words or phrases would you use to describe [BANK]?
Wells Fargo KeyBank USAA CIBC
11%
25%
20%
11%
15%
10%
14%
8%
4%
4%
2%
1%
3%
5%
43%
42%
11%
10%
-4%
-16%
-8%
2%
6%
4%
2%
-9%
-3%
-5%
-12%
-6%
-10%
-17%

9. 17
Overall Results
MOBILE BANKING BENCHMARK STUDY
Change in Associated Brand Attributes
Pre vs. Post Mobile Site Evaluation
For Wells Fargo, brand association with “Difficult” increased. Wells Fargo was also perceived as less “Expensive” after site
interaction.
Expensive
Untrustworthy
Unreliable
Difficult
--Negative
Which of the following words or phrases would you use to describe [BANK]?
-13%
-5%
-5%
-5%
2%
2%
-3%
-4%
4%
16%
-2%
-1%
-1%
0%
0%
1%
Wells Fargo KeyBank USAA CIBC

10. 18
Overall Results
MOBILE BANKING BENCHMARK STUDY
Although fewer participants associated Wells Fargo with the
“Expensive” after evaluating the mobile site, 10% still
considered Wells Fargo to be “Expensive”.
This Wells Fargo participant talks about her perception of Wells
Fargo as “a bank for people with a lot of money.”
00:00
I just fill like it's a bank for people with a lot of money. I don't
feel like it's a type of bank that I would go to. I am a
stay-at-home mom and I am not sure I can even afford having
one of their accounts.
‘‘
’’

11. 19
Overall Results
MOBILE BANKING BENCHMARK STUDY
Net Promoter Score
How likely is it that you would recommend this website to a friend or a colleague?
NPS
-44%
8%
-34%
-22%
60% 24% 16%
26% 40% 34%
52% 30% 18%
38% 46% 16%
Detractor (0-6) Passives (7-8) Promoters (9-10)
KeyBank had the highest overall Net Promoter Score at 8%. Of all the sites evaluated, participants were least familiar with
KeyBank and were more likely to be neutral about the brand. Consistent with other overall site ratings, Wells Fargo also
had the lowest NPS.

12. 23
Task 1 - Research Account
MOBILE BANKING BENCHMARK STUDY
Problems Encountered
Content
Site Performance
Wells Fargo KeyBank USAA CIBC
18%
38%
32%
16%
10%
16%
8%
14%
24%
8%
24%
6%
2%
2%
12%
4%
2%
4%
8%
4%
8%
32%
22%
28%
Difficult to compare accounts
Not enough account information available
Information was confusing / unclear
Site was slow
Encountered errors / techical issues
I did not encounter any problems or frustations
Problems understanding content were fairly common across all the sites, but difficulty comparing content was reported by
about one-third or more of participants on KeyBank and CIBC. While nearly one-quarter of Wells Fargo and KeyBank users
found the information confusing or unclear.
Overall, significantly more Wells Fargo participants encountered issues while researching savings account than participants
on the other three sites.
Which of the following problems or frustrations, if any, did you encounter while
researching savings accounts at [BANK] from your mobile device? [Check all that apply]

13. 26
Task 1 - Research Account
MOBILE BANKING BENCHMARK STUDY
Key Bank First Clicks
Only 4% were successful in finding the minimum balance for
a savings account with the highest interest rate on KeyBank.
Numerous savings accounts are available and several clicks
are required to find the rate information for each account.
When users finally reach the rate and account details page,
the page is not mobile friendly and additional adjustments
are required to be able to view and read the information.
Most efficient path
17%
37%
41%
4%

14. 28
Far Too Much Just Right Too Little
18%
40%
42%
Task 1 - Research Account
MOBILE BANKING BENCHMARK STUDY
Amount of Information Available on the Site
How would you rate the amount of information available on the site?
What information was lacking?
70% of participants thought the CIBC site offered the right amount of information on the mobile site, higher than the
other sites in this study. KeyBank was considered most lacking in terms of amount of information provided.
8%
38%
54%
6%
50%
44%
4%
70%
26%
Wells Fargo Key Bank USAA CIBC
Wells Fargo Key Bank USAA CIBC
“Ability to search that
would then connect to
full site”
“Clearer info about
options on mobile site”
“Information comparing
each account with the
other”
“I would have liked to
see more information
on how long before you
accumulate interest”
“Key details like
minimum amounts and
interest”
“Details like fees for if
your balance drops
below the minimum”
“Information about
their savings account.
There should be a little
more information on
every title/name of the
accounts.”
“A comparison chart
for accounts”

15. Wells Fargo First Clicks
Only 2% of the users took the most efficient path
to find a branch.
10% users said that they clicked on the wrong link
and that there were too many links to click on
before they could find the correct page.
Wells Fargo also had the lowest ease of use ratings
at 58%, the most problems encountered and the
highest time on task at 1:07 minutes.
39
Task 3 - Find a Branch
MOBILE BANKING BENCHMARK STUDY
12%
24%
2%
46%
Most efficient path