My deck of slides on Lean Optimisation, tools, techniques and methods - for you to download and use. Please mail me sullivacATgmail.com with any questions!

2. Naked promotional slide
Tweet me @OptimiseOrDie
• Conversion Rate Optimisation • Contact deflection and online
self service
• A/B and Multi-variate testing
• Site search analytics
• Telephony analytics & tracking
• Site, Page and Campaign
• User centered Design optimisation
• Usability testing • Test design and execution
• Customer Research & Insight • A long usable site portfolio
• Web Analytics • ROI on UX improvements
• Performance, browser and • Over 19 years of slow death
email optimisation in really really boring
meetings
Group CX Manager for Belron® (Autoglass ®)
Unofficial title : Chief evil scientist
Run the ‘League of Optimisers’ on Linkedin

3. Outline
• Lean Optimisation
• Customers drive optimisation
• Analytics tools
• How we build stuff
• How we optimise stuff
• What we found out
• Call tracking
24 Jan 2012

4. Lean UX / Lean Optimisation
• Old but new in the way it’s packaged
• It isn’t really Lean – more sleek
• Combines UCD, Agile PM and tightly integrated teams
• Econsultancy report part 2 – next week
Usability Industry Definition:
“A cross functional, principle driven process
characterized by rituals that predispose teams to high-
quality, high-velocity user experience outcomes”
24 Jan 2012

5. Lean optimisation
Lean startup:
“For a particular vision, test your hypotheses against reality very, very early.
Learn, evolve and repeat.”
Belron definition:
“Using a tightly integrated team and an Agile PM method, we design products
using lots of rapid prototypes, iterations, split testing and consumer feedback
loops. There are no real deliverables, releases or precise products: Only
continually improving experiences with measurably positive outcomes for our
business.”
Short version:
“It makes lots of money and delights customers”
Edison, Dyson and Jobs – for websites.
24 Jan 2012

6. Insight
Research and insight
24 Jan 2012

8. NFL (Normal for Lewisham)

10. Immersion tips
• Wear the magical slippers of customer experience
• Work in the call centre or listen in, regularly.
• Test all the journeys from print, media, display, PPC, organic.
• Exercise your top landing pages or key routes
• What’s the scent trail like?
• Test all the non phone contact stuff in your site (email, forms)
• Test the phone lines people use (quick poll)
• Lovefilm staff packages and incentives (make your own)
I’d like you to immerse yourself:
1. In the complete journeys that people are taking and the product or
service you’re offering
2. In the emotional states they’re in or can get themselves into
3. In all the materials – the website of course but the emails, the phone
on hold music, the SMS messages - absolutely everything about the
customer lifecycle.
24 Jan 2012

11. Our killer optimisation sources…
• Usability testing
• Remote usability testing
• Other usability techniques
• Clicktale
• Surveys
• Customer feedback loops
• Analytics and instrumentation
• BU contacts and market research, local knowledge
• Foolproof and our Optimisation team
• Optimisers in industry, books, websites
• Social networks
• New for 2012? – Higher use of Dynamic surveys, Reviews,
Ratings, Social Proofing, Video
24 Jan 2012

12. Usability testing
• International scope : 35 countries, 19 languages
• Over 300 tests in the last 18 months
• Including 52 mobile handsets, 40 locations and 9 key products
• Usability testing is a vital part of our strategy
• Without this, our products would suck (for long periods)
How does usability testing help?
• Reduces development and ‘get wrong’ time
• Guides product design in the right direction
• Faster iteration of product improvements
• In flight adjustment not ‘like it or lump it’
• There is no IT team or department involved
• We’ve built our development process around this
• Fast design of successful products (mobile, app, website, call centre)
24 Jan 2012

13. Remote usability testing
• What are the advantages here?
• Cheaper, faster than lab based testing
• Belron does this to get large viewing audiences too!
• Participants, Team and Colleagues
• Live audio, video and realtime translation
• Normal test viewing ~3 people. Remote tests = 20+
Use crowdsourced remote testing for:
• Live prototyping and feature tests
• Larger groups can be recruited for volume studies (say a new
postcode lookup function)
• International language support
• Integration between surveys, behaviour and customer
outcomes
24 Jan 2012

14. Budget ideas – no excuse!:
• Low budget prototyping (Mediacam £40, Laptop with webcam)
• Escape from the office prototyping
• Paper prototyping
• Bookshop testing
• Crowdsourced testing – unmoderated online feedback
Please, if you can:
• Link UX work to analytics, NPS and ROI – always test or measure
• Look at your Lifetime value, downstream conversion, return rate etc.
etc. too
“Good in test isn’t always good in pocket”
24 Jan 2012

15. Killer tip #1 : Use clicktale, now
• Vital for conversion rate optimisers.
• Realtime feedback on changes, new product features
• Deep analysis – for example, show me all Google Chrome javascript
errors on the credit card page
• Complete forms analytics (a whole subject area)
• A nice safety net – helps you avoid being fired
• The coolest tool I use in my job – nothing else gets me this stuff
• 15 minutes after publishing new stuff, we’re watching live recordings
• Some examples:
• Q. Where is the fold on the page? A. It’s not a fold, it’s a region
24 Jan 2012

19. Killer tip #2 : End device discrimination
• 33 companies were asked:
• “Measure a key process but segment the conversion or completion
rates by browser type. Tell me what you find.”
• 11 out of 33 found significant holes in checkouts or conversion
funnels. Like lots of money missing, iceberg like. Nobody noticed.
• Broken links, missing continue buttons, form validation, javascript
browser or tag clashes, alignment and visibility issues. Firefox and
Chrome were largest problem areas, not IE6!
• “This store does not support Puma trainers” and the buggy test
• Remember, the % of browser is not the % of revenue, D’oh!
• Stop discriminating – look at your entry traffic (arrivals) split by
browser and test for the main audiences.
• Desktop browser testing - Browsercam
• Mobile device testing - Deviceanywhere
• Noble cause or actually just the Scotsman’s eternal sadness 24 Jan 2012

20. Surveys
• A neglected art
• Put 99% of your time into the design (RIRO)
• Read Catherine Jarrett : http://slidesha.re/mZUeNo
My tips:
• Test at least 3 times with small samples
• Fix problems (e.g. Too many ‘none of the above’ responses)
• Rinse and repeat
• Use someone who has done it before
• Design for outcomes
• General service surveys are great (Lovefilm)
• Are you surveying competitors? Your service? Think about 7 years
of data!
• Where else can we use these wonderful things?
24 Jan 2012

21. Surveys #2
• Dropped basket survey (LF and Paypal/Direct debit driver)
• Remarketing sucks without knowledge. Drops on the floor.
• Bail survey (exit process)
• Behavioural triggers
• Page level feedback
• Site exit survey (satisfaction, what purpose, completed?, why not?)
• Onion layer survey
• NPS – Belron runs a huge program
• UX work = 35% filled responses, 4% dropout, 10% mobile fillout
• VOC products:
• Cool stuff I like : Triggered surveys, crowdsourced feedback, new
product features, show video, adverts or mockups to people
24 Jan 2012

22. Analytics and instrumentation
• For our work, we use 6 tools for optimising outcomes:
Google Analytics Insight, realtime, hypotheses
Speed Trap Advertising response engine
Autonomy Optimost Split and MV testing
Clicktale Session capture, realtime
Addthis Social clickbacks
Call tracking (Various)
So, what about instrumentation?
• We invest time each week on (re)instrumenting the site
• We’re getting richer behavioural data as time goes on
• Slicing and Dicing data becomes simpler and outcome focused
• We measure more phone and contact channel behaviour
• We ask more questions 24 Jan 2012

23. Analytics tips
• Invest in Talent.
• A 35k hire versus a 55k hire is not a 20k saving lol.
• Beware of bull ordure – hard to recruit
• Try my analytics interview tips (email me)
Invest in instrumentation:
• Ringfence development time for continual improvement of
analytics. Give it some love.
• Link behaviours to post conversion event activity (MVT,
Segments)
• Invest in call analytics (on your mobile site, you can do it for
free)
• Drive report vomit out of your business
24 Jan 2012

24. Performance and Availability
• This isn’t a geek thing
• I’ve measured performance impacts
• Dominant factor in mobile conversion
• Big impact on desktop, tablet
• Improves your Google quality score
• Improves your conversion rate
Tip #1 : Google Site Speed
Tip #2 : www.webpagetest.org
Tip #3 : Make development time
Tip #4 : Stop testing mobile on wifi

25. How we build and optimise
From prototype to final product 24 Jan 2012

26. How do we build stuff?
• We don’t know what the final end product will be
• We do have some basics, previous product and insight
• We produce very little documentation
• We start with a clay model (prototype)
• We work out the basic shape
• We then optimise the face in stages
• Once we’re good enough, we go live
• Then we tweak and optimise in place
• The human face of Belron, not a robot
• Emotional and customer centric, not designed by them
• This is the sweet spot
• This is also a key competitive advantage
19 Dec 2011

27. User centred Design
Wireframe
Research Concept Prototype
Analyse Test

28. Multi channel and platform
Washup
Mobile
Web 1
1
UX
Insight Washup
Design
Washup App 1

29. New funnel cycle Analyse
Go live Fix &
10% Improve
Browser
and Device Live 20%
testing
Usability Test and Fix &
Instrument Improve
test
output
Rinse and
Live 50%
repeat
A/B test
live

30. OPTIMISATION
A set of techniques, implemented in
order to influence customer behaviour
towards these outcomes:
• Increased revenue or profit
• Increasing NPS (Customer Sat) scores
• Lower cost for business or contact centres
• Increasing productivity or labour flexibility
• Simply delighting customers
We may fail with 90% of what we try.
The 10% of winners let us shift
behaviour hugely and measurably.

31. What do we use in optimisation cycles?
• Autonomy (for split testing)
• For a budget option, Check Optimizely or Google
• Data warehouse or reporting link (TNC/NPS)
• Clicktale
• Google Analytics
• User Research materials
• Browsercam
• Testing and lots more testing
• Call Centre continual feedback
• Surveys (e.g. did you like X, did you see the Y)
24 Jan 2012

32. Usability & Optimisation process
Final Usability Legal review
Final changes Release build
prototype issues left kickoff
Signoff
Instrument Marketing Cust services
(Legal, Test Plan
analytics review review kickoff
Mktng, CCC)
Instrument
Offline End-End Launch
Contact QA testing
tagging testing 90/10%
Centre
Launch Monitor < 1 Launch
Go live 100% Monitor
50/50% week 80/20%
Analytics Washup and New New test Rinse and
review actions hypotheses design Repeat!

33. A/B and Multivariate tests
How we blend the techniques
24 Jan 2012

35. A/B test
Control Version A Version B

36. Multi variate Pizza
Ingredients Experiment Recipes
=

37. What’s our testing mix?
• A/B testing – Homepage, Landing pages, Funnels
• MVT testing – High traffic landing or Home pages
– Mini (less than 8 variables)
– Midi (8-32)
– Maxi (>32 variables)
And 6 types of specialised testing:
• Funnel testing – bedding in new funnels
• Landing page optimisation
• Cross channel and telephony testing
• Iterative feature and product testing
• Video tests
• Micro testing

38. 6 years of testing on one slide
Less is more – simplicity, comprehension, clarity in design, visibility, space, LESS WORDS
Copy works – Call to action wording, title, headline, strapline, framing – contribute 50-70%
of our test uplift. Pay someone to do this properly, like Sticky Content.
Really big buttony button : Large, prominent, visible, button like CTA
Use goal, verb or action oriented keywords (Fix my glass!, Download now, Get started)
Use text decoration on key persuasive copy (don’t overdo it)
CTA above fold, especially on landing pages, key steps
Click here (Haz clic aqu, cliquez ici) is one of the top 3 button wordings, always. Let me
explain.
Do not split attention on CTA unless necessary. Deprecate other buttons or functions
(Save, Cancel, Print, Back, Help, Buy)
Emotional states : Using emotion, reflecting or mirroring the customer are just as
important as how the checkout form works. If you don’t know what the worries and fears
are from the visitors coming through the funnel, you’re missing a trick.
Show me the value proposition : The what’s in it for me test. (PPC example)
Maintain a scent trail – PPC advert, Organic title and snippet, Landing page, Funnel, Reflect
the user, Maintain the scent, Make use of the gifts you’re given
Think about emotional states and make use of them
Badging : Use badging, testimonials, social proofing and trust symbols
Badging : Be careful about security seals and when you use them. Same with promotion or
visibility of card information – always test.
Images : See next slide
Guidelines are good but every page and site will vary

39. People images that suck…

40. Photo Guidelines
• Groups, vans, stores, head office, • Arms with prop instead – hold a
corporate guff = NO clipboard, rag, tool – I’m working for
• Single person – not groups or even *you*
person + customer (ignoring you) • Female images work best in almost every
• Open, Friendly, Natural, Smiling, country – they slay the guys
Engaged, Alert = YES • Natural, authentic and not ‘model’ or
• Smile best with eye crinkles ‘stock’ image types work best for people
• Uniform – has huge effect, even • Some of our best performing images
wearing branded hat improves would be rejected by advertisers as ‘too
conversion ugly.
• Look straight at viewer or slightly • Want to know more? Contact me or
towards CTA check out the @brainlady
• Plain backgrounds only • Oh, and we also know that genital
• No folded arms, arms on hips, pointing, framing doesn’t work…
scowling, cheesy smiles, hands behind
back, hanging useless by the side,
clasped by groin, fiddling with hands

42. Lean optimisation : The roundup
• Reduced ego and opinion (execs + others)
• Faster time to market
• Measurable ROI
• Increased conversion in the pocket early
• Iterative and rapid product lifecycles
• Faster, smaller, focused, measured releases
• Continual testing = continual improvement
• SLED vs. Agile = faster return, better focus
• Less reliance on anecdotal evidence
• Introduces a testing culture for everyone
• Large cross silo team now gets involved
• Improved £ - but also drives business change

43. Conversion optimisation - Mobile
Why was our mobile stuff rocking?
24 Jan 2012

44. 35
Growth in Worldwide
29.9
Mobile traffic
30
% of all unique visitors
25.8
25
23.9
Nov-11 Jul-11 Dec-10 May-10
20
17.6
16.8
15
13.8
12.8 13
12.7
11.7
10.4
9.8 9.7
10 9.5
7.3
6.5
6.2 6.1
5.8
5.1 5
4.6 4.7
5 4.2
3.6 3.6
3.2
2.7 2.5
2.4 2.3 2.3 2.3
0
Australia USA UK Norway Sweden Spain Italy France Canada Netherlands Germany

45. Our mobile journey…
• Launched Dec 2010
• Mobile web in all countries
• Apple and Android apps in key markets
• Growth areas – iOS, Windows Phone, Android
• Conversion is as high as 30% when including tap to call
• Has delivered over 5M per month in revenue
• Not driven by apps – mobile web 20-300x larger revenue
• Search traffic (intent), a discovery route (SEO and PPC) and an
optimised site = 
• Tablets, though growing, are still only 10-15% of all our visitors
• Everyone needs a highly optimised site

46. Why did it work so well?
• Listened to customers, not our inner • Device aware traffic routing and
geek switching
• Read Luke Wroblewski, Brian • High performance site, even in poor data
Fling, Maximilliano Firtman conditions
• Focused on • Page weight 7-10 Kb payload for each
user, device, context, location, performa page, after cache
nce, emotions. • Performance does not mean poor design
• User centred Design (8 weeks) for fast • Contact options that customers want
execution, high conversion • Cheap mobile targeted PPC and Display
• Total handset compatibility – an advertising
Optimal experience for all device • Total compatibility with their handsets
capabilities, even 10 year old flip
phones (95%+). • Employed a dedicated copy
specialist, Sticky Content
• Deviceanywhere – for testing
• It’s fast – that’s very important for CRO
• Uses HTML5 techniques on supported
phones

47. Killer Tip #3 : Call tracking
How it works:
• Rent a bank of numbers
• Add a dynamic ‘tag’ to your page
• Phone number appears automatically
• Customer calls this number
• Captures the keywords, landing page, converting page, traffic source
• Geo mapping captures customer location
• When the call is hung up, optionally record the outcome code (sale, loss)
• Have a special number for Facebook, SEO, Adwords traffic etc.
• Lets you analyse from web -> phone call -> wait time -> answer ->outcome
• Builds a link between web and call, finally

48. Why use call tracking?
• Claim your conversions!
• You can’t work out your advertising effectiveness, if you don’t include
phone
• If you don’t measure phone, your A/B tests may be wrong
• You can capture all your web AND phone, in one easy interface
• Allows you a multi-channel view of all your contact sources
• Lets you record offline campaigns as well with specific numbers (print, TV
etc.)
• You can experiment with routing, messages, on hold music – much more
easily
• If you can’t afford lots of phone numbers, this is an easy way to get
tracking
• Puts this in the hands of small businesses

49. Company Website Coverage
Who?
Mongoose Metrics* www.mongoosemetrics.com UK, USA, Canada
Ifbyphone* www.ifbyphone.com USA
TheCallR* www.thecallr.com USA, Canada, UK, IT, FR, BE, ES, NL
Call tracking metrics www.calltrackingmetrics.com USA
Hosted Numbers www.hostednumbers.com USA
Callcap www.callcap.com USA
UK, SE, FI, NO, DK, LT, PL, IE, CZ,
Freespee* www.freespee.com
SI, AT, NL, DE
Adinsight* www.adinsight.co.uk UK
Infinity tracking* www.infinity-tracking.com UK
Optilead* www.optilead.co.uk UK
Switchboard free www.switchboardfree.co.uk UK
Freshegg www.freshegg.co.uk UK
Avanser www.avanser.com.au AUS
Jet Interactive* www.jetinteractive.com.au AUS
* I read up on these or talked to them. These are my picks .

50. Autoglass UK – Putting it together
• 12.4% from User centred Design
• 12.5% increase from Multi Variate Testing
• 3.5% increase in downstream conversion rate
• 4% increase in online conversion from mobile optimised site
• Reduced call time (1 second = > 10k per year)
• Faster online booking time
• 5.5% increase in NPS (Customer sat) score
• Customer delight – feedback. Less questions and worries
• Total increase in conversion (measured) = 32%
• Actual increase in conversion = 41%
• NO additional marketing spend needed
• Web Channel : ROI positive within 2 weeks
• Worldwide ROI : 15% increase in conversion during 2011

51. More reading. Slides and resources on slideshare.net
Email : sullivac@gmail.com
Twitter : @OptimiseOrDie
: linkd.in/pvrg14
Slideshare : slidesha.re/nlCDm6