So you’ve read the book and you know everything there is to know about CRO but nobody is listening! It doesn’t matter how good you are because without buy in and a data-driven testing culture embedded across your digital organisation you will never reach your full potential. I’ll show you how one of the oldest banks in the world went from a standing start all the way through to testing and optimisation Superstars.

1. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
Thomas Barker
Data and Analytics Lead: Digital & Optimisation
Thomas.Barker@rbs.co.uk // @tomabarker

2. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.

3. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.

4. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.

5. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
Culture - Pronunciation: /ˈkʌltʃə/
“The ideas, customs, and social behaviour of a
particular people or society.”

6. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
You should test that.
Can/Did we test that?

7. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
• Build the brand
• Get buy-in from the Executive Tier
• Show the value quickly and keep telling people
• Humour is your friend
• Don’t hide away

8. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.

9. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
 Serving customers in Scotland since 1727
 15 million customers in the UK
 ~ 90,000 staff
 Our aim is to be the Number 1 Bank in the UK
for Service, Advocacy and Trust
© 2015 Royal Bank of Scotland Group. All Rights Reserved.

10. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
FIREWORK CULTURE
Light, run away, everybody claps
© 2015 Royal Bank of Scotland Group. All Rights Reserved.
Our Digital Looked Like This …
 Changes made by gut feel
 Backlog of improvements to the digital
experience stuck in a queue
 Tweaks masked by larger updates
 Too much emotional investment to fail
 Not enough data to succeed
 Celebrated the launch and not success

11. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.

12. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
Traffic
· Despite visitors to B&C remaining largely static this month, the number of visits has reduced by almost 400k (6.2%) – something
that is largely a result of reduced traffic from direct sources (most likely users of Bankline).
· The reduction has also been partially contributed to by activity on the retail site, which contributes 20% of all business traffic
indirectly. It can be difficult to discern the impact of retail's marketing spend on B&C traffic directly, but we can work it out by
observing paths into public web from retail pages in sessions with a retail campaign code - there were 15k fewer visits to B&C from
retail-branded PPC activity in the month, due to an overall reduction in their marketing in this area.
Sales
· Various optimisations to the MTA Terms &Conditions pages have come together to improve site commerciality by 15%. However, the
success in moving customers from the T&C’s page into the form has also resulted in a lower overall form conversion rate (down 3.7%)
– further refinement of the MTA application form is necessary to realise the full benefit of optimisation activity further upstream.
· Display (external banners) activity started in early December has resulted in an influx of low quality but high volume start-ups
traffic. Although we may see some increase in overall sales volumes, this type of referral does not help the Campaign start rate,
which has fallen dramatically from a high of 40% to just 3.8%.
Marketing
· The NatWest PPC campaign is targeting generic lending phrases (small business loan, business loan etc…) and our acquisition
through this channel is getting better as a result - up by 40% to 9k. However, the highly competitive nature of the lending market
for new customers is shown in the relatively low conversion rate of just 2.5% to the completion of the enquiry form.
· HSBC is in the enviable position of having the top spot on the natural section of the "small business loan" phrase. This means
that, currently, HSBC will always be featured prominently in search results without having to bid for position. Conversely, we have
disabled crawler access to the RBS campaign landing page, which means we don't let it be known to Google that it's available. Due in
part to this, RBS is only featured on results page 5, below all other main competitors. We should consider allowing Google to index
the campaign landing pages so that natural traffic can apply using this well-optimised landing page and we can get back to near the
top of the rankings for natural search. This would provide a short-cut for customers to reach the application stages, without
needing to traverse the long and winding road through multiple search results pages and through the public web navigation
hierarchy.
Our Analytics and Optimisation Looked Like This …

13. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
Traffic
· Despite visitors to B&C remaining largely static this month, the number of visits has reduced by almost 400k (6.2%) – something
that is largely a result of reduced traffic from direct sources (most likely users of Bankline).
· The reduction has also been partially contributed to by activity on the retail site, which contributes 20% of all business traffic
indirectly. It can be difficult to discern the impact of retail's marketing spend on B&C traffic directly, but we can work it out by
observing paths into public web from retail pages in sessions with a retail campaign code - there were 15k fewer visits to B&C from
retail-branded PPC activity in the month, due to an overall reduction in their marketing in this area.
Sales
· Various optimisations to the MTA Terms &Conditions pages have come together to improve site commerciality by 15%. However, the
success in moving customers from the T&C’s page into the form has also resulted in a lower overall form conversion rate (down 3.7%)
– further refinement of the MTA application form is necessary to realise the full benefit of optimisation activity further upstream.
· Display (external banners) activity started in early December has resulted in an influx of low quality but high volume start-ups
traffic. Although we may see some increase in overall sales volumes, this type of referral does not help the Campaign start rate,
which has fallen dramatically from a high of 40% to just 3.8%.
Marketing
· The NatWest PPC campaign is targeting generic lending phrases (small business loan, business loan etc…) and our acquisition
through this channel is getting better as a result - up by 40% to 9k. However, the highly competitive nature of the lending market
for new customers is shown in the relatively low conversion rate of just 2.5% to the completion of the enquiry form.
· HSBC is in the enviable position of having the top spot on the natural section of the "small business loan" phrase. This means
that, currently, HSBC will always be featured prominently in search results without having to bid for position. Conversely, we have
disabled crawler access to the RBS campaign landing page, which means we don't let it be known to Google that it's available. Due in
part to this, RBS is only featured on results page 5, below all other main competitors. We should consider allowing Google to index
the campaign landing pages so that natural traffic can apply using this well-optimised landing page and we can get back to near the
top of the rankings for natural search. This would provide a short-cut for customers to reach the application stages, without
needing to traverse the long and winding road through multiple search results pages and through the public web navigation
hierarchy.
Our Analytics and Optimisation Looked Like This …

14. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.

15. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
 Connected to your audience
 Experiment with new content
 Pull back if things don’t work
 Appreciate segments and changing set list
 Do it all yourself
 Work close to real time
BE FEARLESS
1515© 2015 Royal Bank of Scotland Group. All Rights Reserved.

16. © 2015 Royal Bank of Scotland Group. All Rights Reserved.
Journey Manager aka DJs and Producers. Who are they?
16
Data & Analytics
Journey
Managers
COLLABORATION

17. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
First Steps: Gain Control and Insight
Implement Journey Managers and CMS
Experiences built from sandpit of pre-built elements
1 Implement Tag Management –
Single View of Journey
2
Complex series of platforms makes up digital experience

18. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
Democratise the data - 110 Live data
dashboards, Clicktale, etc
Always on and self-serve data. Spot opportunities and struggles
3 Help answer the why!
Begin optimisation.
4
First Steps: Gain Control and Insight

19. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
It’s a Dilemma
RESOURCES
ANALYSIS
TESTING
OPTIMISATION

20. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
The Solution

21. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
Share the Learning, Share the Success
TEST
LEARN
COLLABORATE
© 2015 Royal Bank of Scotland Group. All Rights Reserved.

22. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
Transparency in Analytical Pieces, Insight and Results

23. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
Superstar DJs dramatically increased optimisation of the digital
experience
© 2015 Royal Bank of Scotland Group. All Rights Reserved.
# of Optimisation
Specialists:
H1 2014
2
Test Completed:
2
Size of Digital
Analytics: 6
# of Optimisation
“Specialists”:
2015
70
Test Completed: 400
Size of Digital
Analytics: 10

24. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
Own the Moment
© 2015 Royal Bank of Scotland Group. All Rights Reserved.

25. © 2015 Royal Bank of Scotland Group. All Rights Reserved.
1. PEOPLE:
DJs need the right
equipment and training
25

26. © 2015 Royal Bank of Scotland Group. All Rights Reserved.
“Train the Trainer” Approach
26
Start with nucleus
of Experts – Expand
through internal
training academies

27. © 2015 Royal Bank of Scotland Group. All Rights Reserved. 27
“Think and speak the same language as the
business – not like an analyst/optimiser/guru”
Technical skills are important, but you need to have a business head on

28. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.

29. © 2015 Royal Bank of Scotland Group. All Rights Reserved.
Creating a Superstar DJ: Continuous Training, Growing the Team
BEDROOM DJ
relies heavily on
Producer
WEDDING DJ
can manipulate
basic reports.
Understands
data and
insights
CLUB DJ
Creates
hypotheses
based on data &
insights and
creates tests.
SUPERSTAR
DJ
self solves
29

30. © 2015 Royal Bank of Scotland Group. All Rights Reserved.
2. TRULY UNDERSTAND
JOURNEYS
30

31. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
We Used to be Organised as if Customers Behaved Like This…
RESEARCH ADOPT SERVICE
Web
Telephony
Branch

32. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
The Reality is a Bit More Like This …
RESEARCH SERVICE
Web
Mobile
Branch
Social
Web Chat
Telephony
ADOPT

33. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
Do you really understand Key Journeys? Why are they happening?
RESEARCH SERVICE
Web
Mobile
Branch
Social
Web Chat
Telephony
ADOPT

34. © 2015 Royal Bank of Scotland Group. All Rights Reserved.
3. PROGRAM AWARENESS
AKA THE PR FACTORY
34

35. © 2015 Royal Bank of Scotland Group. All Rights Reserved. 35
Get Executive Buy In
To get executives on-board, solve their problems!!

36. © 2015 Royal Bank of Scotland Group. All Rights Reserved. 36
Shameless Self-Promotion: Managers must be Evangelists
Identify the opportunity
Build the test
Show results
Give them bragging rights
Understand their challenges

37. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
The Proposition
• We recently launched a new
current (checking) account
reward add on proposition
• Costs £3 a month
• 3% back on all bills setup as
Direct Debits (Automated
Clearing House).
• A bank giving you money

38. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
Which Test Won: Online Banking Logout Page
Incumbent
1 2 Challenger
• 296% increase in
conversion
• 300 extra
upgrades a day

39. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
So Who Did That Particular Test?

40. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved. 40
Not Who You Would Expect

41. © 2015 Royal Bank of Scotland Group. All Rights Reserved. 41
Landing the Exco Record Deal

42. © 2015 Royal Bank of Scotland Group. All Rights Reserved. 42
The Live Tour

43. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
Visit your Frontline!!

44. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
Have a PR schedule. I bet your company does, why don’t you?

45. © 2015 Royal Bank of Scotland Group. All Rights Reserved.
Engage Your Audience … and Watch How They Engage
© 2015 Royal Bank of Scotland Group. All Rights Reserved.
“It’s cool and playful… I look
forward to it every week.”
Ankit Chhajer, Journey Manager
“…the way you’re bringing this to life for the team is
nothing short of brilliant!”
John Montgomery, Head of Digital Sales
“Love it!”
Katie Cassidy, Head of
Digital Service
“Loving your hard work!”
Stuart Haire, Head of Direct Bank
“Love this... Can we get SSDJ for
my team?”
Wincie Wong, Lead Journey Manager (CT)

46. © 2015 Royal Bank of Scotland Group. All Rights Reserved.
4. KEEP SHARING
46

47. © 2015 Royal Bank of Scotland Group. All Rights Reserved.
Be Willing to Share.
Know how to say No.1
Keep embedding the self
serve approach.2
47© 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

48. © 2015 Royal Bank of Scotland Group. All Rights Reserved. 48
Share the Learnings
THE MANTRA:
Spot opportunities.
Test content.
Own the experience.
Share the Learnings

49. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
So What Next?
© 2015 Royal Bank of Scotland Group. All Rights Reserved.

50. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
We use data to decide the best message Meanwhile in the app….

51. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
Sometimes though our customers just tell us , through their
actions, what they are interested in Meanwhile in the app….

52. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
It’s up to us to react and continue that conversation in
realtime Meanwhile in the app….

53. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
But which is best?

54. © 2015 Royal Bank of Scotland Group. All Rights Reserved.
Bigger, Better, Faster

55. © 2015 Royal Bank of Scotland Group. All Rights Reserved.
Common Sense is so rare
these days it should be
classed as a Superpower

56. © 2015 Royal Bank of Scotland Group. All Rights Reserved. 56
Rubber Band Principal

57. © 2015 Royal Bank of Scotland Group. All Rights Reserved.© 2015 Royal Bank of Scotland Group. All Rights Reserved.
Thank You
© 2015 Royal Bank of Scotland Group. All Rights Reserved.