This document summarizes a case study on using customer experience management tools at the Rotman School of Management, University of Toronto. It describes how Rotman implemented tactics like A/B testing, personalization, and analytics over several campaigns from May to September. The results showed reductions in bounce rates and increases in time on site and pages per visit. For example, bounce rates decreased 23% and time on site increased 14% after implementing profile-based personalization. The document concludes by recommending engaging all stakeholders, using analytics to guide strategy, and focusing initial efforts in high traffic or impact areas.

1. A Case Study
Prepared by: Randy Woods

2. • Introductions
• The Changing World of Customer Experience
Management
• Barriers to Customer Experience Management Success
• Case Study: Rotman School of Management, University
of Toronto
• Concluding Thoughts, Questions and Answers

3. •
President, Nonlinear Creations
•
Helping organizations figure out what to do
about this thing called the internet since 1995
•
16 years experience with data-driven online
marketing tactics
•
Past 4 years, has lead Nonlinear’s engagement
with Sitecore Digital Marketing System
•
@randywoods

8. Something fundamental is shifting in the
relationship between marketer and marketee
The power has shifted to the victims recipients of
marketing activities

9. • Forrester Research – “WCM for Digital Customer Experiences”
• Gartner Group – “CRM Multichannel Campaign Management” +
“Web Content Management”
• Digital Clarity Group – “Web Content and Customer Experience
Management”

11. •
Assess Visitor Engagement
•
Explicit Personalization
•
Visitor Profiles
•
Engagement Plans
•
A|B and Multivariate Testing
•
Lead Scoring
•
Sales Enablement
•
Mobile, Social Integration…….
•
•
Not all visitors are equal – score them differently
Put visitors in their place
Find out what works
Rules-based Implicit Personalization
Target content based on geography, organization…
Persona- or Profile-based Implicit
Personalization
Target specific content to specific personas
Target content based based on data in your CRM
Automate marketing post initial contact
Rate a lead based on behaviour
Provide sales with insight

12. • Rules-based implicit
personalization
• Profile-based implicit
personalization
• A|B testing
• Explicit personalization
• Engagement plans
• Sales enablement
• Lead scoring
• Engagement value

13. e
•
A|B testing
•
Engagement value
•
Multivariate testing
•
Engagement plan
•
Rules-based implicit
personalization
•
Persona-based
implicit
personalization
•
Lead scoring
•
Sales enablement
•
Explicit, CRM-driven
personalization
•
Engagement plans
•
Sales enablement

15. • The sheer array of tactics available can lead to confusion on
where to start
• The ease of implementation means campaigns launch
without an understanding of what success might mean
• Availability of analytic skills or access to resources with
analytic capabilities
• Lack of senior management understanding of the potential
– and the cost – of CEM
• Technical barriers

17. •
The Outcome:
- A 6-18 month DMS roadmap of DMS tactics ordered to deliver value
-
•
quickly while building for long term success
1-3 tactics implemented and running
The Process:
-
Deep review of existing analytics
Stakeholder interviews to identify marketing needs and gaps
Analysis and identification of quick wins
Implementation of quick wins

18. Rotman Programs – Engagement Level
Bounce Rate
Pages per visit
Mfin
Mfin
EMBA
EMBA
EP
EP
MBA
MBA
PT MBA
PT MBA
PhD
PhD
0%
10%
20%
30%
40%
50%
60%
70%
0
1
2
3
4
5
6
7
8
9

19. Home page visitors going to
fees and expenses - MBA - by
visit number
Fourth+ visit
Third time visitors
Application deadlines from
MBA Fulltime page
Fourth+ visit
4.03%
Third time visitors
6.94%
Second time visitors
8.65%
New Visitors
10.11%
Admissions criteria from
MBA Fulltime page
Third time visitors
Second time visitors
New Visitors
5.67%
Second time visitors
New Visitors
Fourth+ visit
6.54%
7.16%
10.53%
11.09%
13.49%
Interests and information needs vary with number of visits
5.47%
4.42%

20. May
Initial Campaigns
A|B test for EP 1
A|B test for EP 2
Ambassador Personalization
Mid-Term Campaigns
Geo-IP personalization
Target work permit info on home page
Target international campus info on home page
Highlight International applicants link
Show Toronto-prospects part time info on home page
Visit number personalization
Home page personalization
Company-based personalization
EMBA prospect matching to company alumni
EMBA prospect matching to alumni from vertical
EP prospect matching to companies where custom training is provided
A|B Tests
EP test set A
EP test set A.1
EP test set B
EP test set B.1
Ambassador page test set A
Ambassador page test set A.1
Home page test set A
Home page test set A.1
Full Time MBA test set A
Full Time MBA test set A.1
PT MBA test set A
PT MBA test set A.1
Application Process A
Application Process A.1
EMBA test set A
EMBA test set A.1
Engagment Plans
EP engagement plan one
EP engagement plan two
EP engagement plan three
Engagement Value
Engagement Value implementation
June
July
August
September ctober NovemberDecemberJanuary
O
February March
April
May
June
July
August
September

22. Test
A
Test
B

23. Test a
Test b

25. Exit rate – from student services page
Avg. time on page - seconds
13%
All Visitors
All visitors
From China
From China
-43%
0.0%
5.0%
10.0%
15.0%
No before and after tracking
– best available stand in metric
20.0%
25.0%
0
5
10
15
20
25
30
35
40

28. Bounce Rate
45%
40%
-23%
35%
30%
Visit Duration (seconds)
Pages per visit
350
25%
20%
4.5
23%
300
4
15%
250
10%
3.5
200
5%
3
150
0%
Pre-personalization
Post-personalization
14%
2.5
100
2
50
1.5
1
0
Pre-personalization
Post-personalization
0.5
0
Pre-personalization
Post-personalization

31. Can I get
in and
how
much
does it
cost?

32. Why
should I
choose
Rotman?

33. What will I
learn and from
who?

34. Call to Action

35. New Users
PrePersonalization
PostPersonalization
Second Visit
Third Visit
Fourth+ visit
49%
44%
42%
53%
51%
45%
45%
47%

36. New Users
PrePersonalization
PostPersonalization
Second Visit
Third Visit
Fourth+ visit
49%
44%
42%
53%
51%
45%
45%
47%
Only 4+ Visit Figure is statistically valid

37. • Engage every person who has responsibility for business
outcomes of the website
• Use analytics to guide your roadmap:
- Segment current traffic by each of the dimensions against which
you can target content – geography, visit frequency, company
- Look for significant behavioural differences
• Focus initial efforts on highest reward areas:
- One lead gen form used by entire organization?
- The homepage! Traffic dwarfs your landing pages.

38. @nonlinear_tweet
@randywoods
Blog.nonlinearcreations.com