1. Optimization Is A Journey

2. 7 Habits of Highly
Effective Personalisation
Organisations
Dan Ross
Managing Director, Optimizely ANZ
dan.ross@optimizely.com

4. HABIT 1
Create a
Vision

6. “The best vision is insight.”
– MALCOM FORBES

7. Habit 1 Takeaway:
Create a Vision

8. HABIT 2
Experimentation Maturity

9. ARE WE READY TO STEP FORWARD?
FORRESTER’S PERSPECTIVE
*Source: Forrester’s Q3 2015 Global Online Testing Platform Customer Online Survey
Dimensions
of continuous
optimization
Online testing is applied
mostly to the “explore”
and “buy” phases of the
customer life cycle
Online testing is
applied
mostly to websites
Online testing practices are
mostly executing only A/B
tests
A minority (i.e., 30% or fewer) of
customer interactions are included in
online testing*
Opportunity for improvement

10. ARE WE READY TO STEP FORWARD?
FORRESTER’S PERSPECTIVE
*Source: Forrester’s Q3 2015 Global Online Testing Platform Customer Online Survey
Dimensions
of continuous
optimization
Online testing is applied
mostly to the “explore”
and “buy” phases of the
customer life cycle
Online testing is
applied
mostly to websites
Online testing practices are
mostly executing only A/B
tests
A minority (i.e., 30% or fewer) of
customer interactions are included in
online testing*
Opportunity for improvement
MATURE OPTIMISATION PROGRAMS
• Do more complicated tests than A/B
• Test through more than just a few pages
• Are segmenting analytics

11. ARE WE READY TO STEP FORWARD?
OPTIMIZELY’S MATURITY MODEL
INTERESTED INVESTED INTEGRATED INGRAINED
VALUE
Culture
Process
Strategy
Development

12. INTERESTED INVESTED INTEGRATED INGRAINED
VALUE
Culture
Process
Strategy
Development
• Inconsistent access
to resources
ARE WE READY TO STEP FORWARD?
OPTIMIZELY’S MATURITY MODEL
MATURE OPTIMISATION PROGRAMS
• Are comfortable pushing boundaries
• Have processes and teams in place
• Speak language of testing

13. LEADING
INDICATORS
Experimentation Success
VELOCITY
The volume of experiments being run, the
reach of personalisation campaigns.
Throughput:
# of experiments per property per
month/week.
AGILITY
The degree that the experimentation
program acts on results.
Iteration:
The % of experiments put into production
and iterated upon.
EFFICIENCY
The efficiency that experiments get
through production cycle
Drag:
Average hours spent
redeveloping due to QA
QUALITY
The average likelihood that an
experiment will produce
business impact
Impact Rate:
% generating meaningful result
OPERATIONAL METRICS FOR EXPERIMENTATION

14. LEADING
INDICATORS
Experimentation Success
VELOCITY
The volume of experiments being ran,
the reach of personalization
campaigns.
Throughput:
# of experiments per property per
month/week.
AGILITY
The degree that the experimentation
program acts on results.
Iteration:
The % of experiments put into
production and iterated upon.
EFFICIENCY
The efficiency that experiments get
through production cycle
Drag:
Average hours spent
redeveloping due to QA
QUALITY
The average likelihood that an
experiment will produce
business impact
Impact Rate:
% generating meaningful result
OPERATIONAL METRICS FOR EXPERIMENTATION
MATURE EXPERIMENTATION PROGRAMS
• Are high throughput
• Develop efficiently (business as usual!)
• Get consistent wins

15. Habit 2 Takeaway:
Experimentation
Maturity

16. HABIT 3
Assemble
Your
Dream Team

17. DISCOVERY IMPLEMENTATION PLANNING PRODUCTION REPORTING
PERSONALISATION PLAYBOOK
END-END PROCESS + MILESTONES

18. CORE PERSONALISATION TEAM
SKILLSETS & TEAM ROLE
Executive Sponsor Project Manager Technical Lead Developer Content

19. make
a slack channel
make
the time

20. Habit 3 Takeaway:
Assemble Your
Dream Team

21. HABIT 4
Enrich Your
Perspective

22. YOUR
Team
Status Quo:
Tech: current capabilities and limitations
People and Process
Audience Strategy
Look Internally
Your Systems
Your Analytics
Your Personas
Your Competitors
Your Strategy
Future States:
Potential capabilities
Audience Proposal
Use Cases
YOUR TEAM’S TASK
GATHER INTELLIGENCE: LOOK INWARD
1

23. YOUR
Team
Validation and
Alternate Perspectives:
Tech: Potential capabilities
People and Process: Alternate Approaches
Audience Strategy
Consult
External Experts
Vendors
Consultants
Agencies
Analyst Reports
Future States:
Potential capabilities
Audience Proposal
Use Cases
2
YOUR TEAM’S TASK
GATHER INTELLIGENCE: LOOK OUTWARD

24. YOUR
Team
Status Quo:
Tech: current capabilities and limitations
People and Process
Audience Strategy
Validation and
Alternate Perspectives:
Tech: Potential capabilities
People and Process: Alternate Approaches
Audience Strategy
Consult
External Experts
Vendors
Consultants
Agencies
Analyst Reports
Look Internally
Your Systems
Your Analytics
Your Personas
Your Competitors
Your Strategy
Future States:
Potential capabilities
Audience Proposal
Use Cases
YOUR
Brief
3
YOUR TEAM’S TASK
GATHER INTELLIGENCE: CONSOLIDATE

25. YOUR
Team
Status Quo
Validation and
Alternate Perspectives
Consult
External
Experts
Look
Internally
Future States
YOUR
Brief
3
YOUR TEAM’S TASK
GATHER INTELLIGENCE: CONSOLIDATE

26. Habit 4 Takeaway:
Enrich Your
Perspective

27. HABIT 5
Create Your
Audience
Strategy

28. Recency & Frequency
Cross-sells & Up-sells
Value Propositions
START BY EXAMINING YOUR BUSINESS
STRATEGY
Propensity Models
Customer Journey Model
Price Sensitivity

29. LAYER ON MORE AUDIENCES
LEFT- & RIGHT-BRAIN
PERSONAS ANALYTICS

30. WHAT TECHNICAL SIGNALS CAN WE LEVERAGE?
CONNECT CONCEPT TO TACTIC
Viewed 2 Products, Didn’t Buy
Keyword contains ‘discount’
Most frequently viewed
category
DMP + Uploaded Lists
Abandoned Checkout
Data Warehouse (Customer
ID
Geo-Targeting)
Came from Ad Campign = Gift
Technical
Signal Consideration-Stage
Wants a discount
Preference for a specific
product type
High-Propensity
Needs a push
VIP Member
Urban Location
Shopping for a Gift
Audience
Characteristic

31. PRIORITISE, PRIORITISE, PRIORITISE
PURSUE VARIETY OF AUDIENCES, MAXIMISE REACH/QUALITY
Obvious Need
Large
Need for Creativity
Granular
Visitor Cohort; New,
Returning, Active, Loyal
Large Geos; Coastal
Urban, State, Key Cities
Browsed Twice;
Product Category
Past Purchasers
Second Priority

32. Habit 5 Takeaway:
Create Your
Audience Strategy

33. HABIT 6
Unify

34. View of the
Customer
CONNECT YOUR DATA
HOUSEKEEPING BEFORE TECHNOLOGY

35. Everyone has to work together
for personalisation to work for you

36. Habit 6 Takeaway:
Unify

37. HABIT 7
Crawl Before
You Walk

38. PHASED INTEGRATION OF PERSONALISATION
CRAWL, WALK, RUN
0-12 weeks
Build
Phase
1
months 12-24
Build
Phase
3Build
Phase
2
months 3-12

39. Platform Implementation
Simple Audiences
Starter Campaigns,
Limited Integration of
Testing + Personalisation
Phase 2 Planning
REACH: 0-15%
PAGES: 1-3; only most critical ROI points
#
CAMPAIGNS: 2-5
AUDIENCES: Natively available, simple, large, simple conditions;
Metro, Single Behaviours
TACTICS: Modules (lightboxes), image swaps, little testing
0-12 weeks
Buil
d
Phase
1
PHASED INTEGRATION OF PERSONALISATION
CRAWL, WALK, RUN

40. Integration with 1st & 3rd
Party Data
More Campaigns
Integration of testing &
Personalisation workflows
More advanced use cases
Phase 3 Planning
Buil
d
Phase
2
months 3-12
PHASED INTEGRATION OF PERSONALISATION
CRAWL, WALK, RUN
REACH: 30-60%
PAGES: Multiple campaign/audiences on top ROI pages
#
CAMPAIGNS: 10-20 ongoing campaigns
AUDIENCES: Target intersecting audiences, 3rd & 1st party data
used, more and complex behaviours
TACTICS: Experiments drive campaign execution and iteration

41. Full system integration
Ongoing improvement
New audience strategy
Use cases continually iterated
Web personalisation data feeds
email and ad deployment
Buil
d
Phase
3
months 12-24
PHASED INTEGRATION OF PERSONALISATION
CRAWL, WALK, RUN
REACH: 75-100%
PAGES: Most pages, multiple elements per page
#
CAMPAIGNS: 25+ ongoing personalisation campaigns iterated on
AUDIENCES: Old audiences iterated, new granular audiences
TACTICS: Fully expressive strategy

42. Habit 7 Takeaway:
Crawl Before You Walk

43. Experimentation Maturity
Create a Vision
Assemble Your Dream Team
Enrich Your Perspective
Create Your Audience Strategy
Unify
Crawl Before You Walk