MAKING YOUR ANALYTICS TALK BUSINESS Aligning your analysis to the business is fundamental for all types of analytics (digital or product analytics, business intelligence, etc) and is vertical- and tool agnostic. In this talk we will build on the discussion that was started in the previous meetup, and will discuss how analysts can learn to derive their stakeholders' expectations, how to shift from metrics to "real" KPIs, and how to approach an analysis in order to create real impact. This session is primarily geared towards those starting out into analytics, practitioners who feel that they are still struggling to prove their value in the organization or simply folks who want to power up their reporting and recommendation skills. If you are already a master at aligning your analysis to the business, you're most welcome as well: join us to share your experiences so that we can all learn from each other and improve! Bios: Eliza Savov - Eliza is the team lead of the Customer Experience and Analytics team at Clicktale, the worldwide leader in behavioral analytics. She has extensive experience working with data analytics, having previously worked at Clicktale as a senior customer experience analyst, and as a product analyst at Seeking Alpha.

1. Making Your Analytics
Talk Business
Eliza Savov
BI and Analytics Demystified Meetup
July 21, 2019

2. A Short Intro…
• Manager CX Research & Analytics at Clicktale
• Clicktale (+ Contentsquare) is the leader in customer experience
analytics (in-page analytics, behavioral analytics)
• We help fill in the missing parts into our clients’ overall understanding of
how people use their digital assets: the ‘why’ and ‘how’ into the ‘what’.
• Some of our clients: The Home Depot, Costco, Microsoft, Disney, Fidelity,
Amex, Royal bank of Scotland, B&Q, Vodafone Ziggo, Ancestry…)

3. Today’s Topics
• Get the Business Context
• How do you ensure your measurement framework is based on real
business needs?
• Collect the right data
• Which data points will help you answer your questions?
• How to provide data that drives action?
• Whether you call it a report or an analysis, how do you deliver results
that matter?

4. 3. Results
2. Data Collection
1. Business Requirements

5. Get The Business Context

6. Getting the Biz Needs is an Ongoing Process
Get the
Goals
New
feature
Get the
Goals
New Business
Initiative
Get the
Goals
New
Campaign

7. The Stakeholder and the Analyst
“Don’t give me
things I already
know, or just
data!”
“Don’t expect
actionable
insights if you
can’t even tell
me what your
goals are.”
Inspiration: Tim Wilson
Images: Man head by Oksana Latysheva from the Noun Project

8. The Answer to “The KPI Question”…
“Uhmm
well…..”
“What are
the KPIs for
this campaign
/feature?”

9. …Might Disappoint You
Magic questions.
1. What are we trying to
achieve?
2. How do we know if we’ve
done that?
Visits
Purchases
Registrations
Drive Awareness to Y
New vs Returning
AOV
Clicks on X CTR
Time on Site
Bounce Rate
Time on Page
Scroll Reach

10. Communication Is The Underlying Issue
KPI --------------------------------
Modified from: Dilbert.com

11. Translating the Request to KPIs
Very broad
idea
or formulation of
what they are
interested in
Source: Tim Wilson
Concrete goals
that match
the overall business
context
KPIs

12. Example: What the Stakeholder Asks
“We just recently redesigned our
Search Result Pages and
introduced some new logic to the
filters, as well as added some new
ones. How do people interact with
the page now?”

13. If The Analyst Asks For KPIs…
• Filter usage
• Scroll reach
• Engagement Time
• Time on Page
• Exit Rate
• Clicks on results

14. Ask in Plain <insert_language_here>
Magic questions.
1. What are we trying to
achieve?
2. How do we know if we’ve
done that?
1. What are we trying to achieve?
2. How do we know if we’ve done that?
Source: Tim Wilson

15. “What are you trying to achieve with
the redesign project?”
• Improve product findability via Search
especially on pages with more than X results.
• Improve visibility of key filters that in the
old design were below the fold and hard to
find
• Introduce new filter options that we were
lacking (because customers complained about
it)

16. Derived KPIs
• Application rate of any filter (→ improved filter visibility)
• CTR on results with filters applied, when initial results >=X (→
improved logic & product findability)
• Application rate of the new filters (→ confirm interest)

17. Magic questions.
1. What are we trying to
achieve?
2. How do we know if we’ve
done that?
1. What are we trying to achieve?
2. How do we know if we’ve done that?
Source: Tim Wilson
Ask in Plain <insert_language_here>

18. Derived KPIs
• Application rate of any filter (→ improved filter visibility)
• CTR on results with filters applied, when initial results >=X (→
improved logic & product findability)
• Application rate of the new filters (→ confirm interest)

19. Targets!

20. Targets Turn a Metric into a KPI
• Who owns the target setting?
• What if the stakeholder refuses to set a target?
• Use internal benchmarks / historical data
• Suggest a number and get a reaction ;)
Source: Tim Wilson

21. Understand Your Stakeholders
Ask the
two open
questions
Translate the
answers into
metrics
Turn those
metrics into
KPIs by
setting
targets

22. The Data Collection

23. You Already Have Tons Of Data

24. …And It’s Growing
Source: Chiefmartec

25. How Much Data Should You Collect & Retain
• The core data describing your business – ongoing.
• Ad-hoc: Specific metrics for specific projects (experiments, new
features).
• Make sure you have the right data to measure those KPIs you
derived.

26. Translate KPIs Into Tagging Requirements
• Application rate of any filter (→ generic event)
• CTR on results with filters applied, when initial results >=X (→
event for product clicks, eVar for filtered page, event for
number of results)
• Application rate of the new filters (→ event per filter)

28. The Process of Data Collection Can Be
Very (Too?) Time-Consuming
• New tagging / event creation for new features affects your time
to deliver and needs to be taken into consideration.
• The process depends on your autonomy to create
customization/tagging/modeling.
• Data collection and cleaning should not be the core of your job.
Analysis is.

29. The Analysis Process

30. BUSINESS ANALYSIS
• Hypothesis generation and
validation as a result of a
business question being
asked.
Two Types of Output
REPORTING
• Ad-hoc quantifications
• Quick data checks
• Performance measurement
• Recurring reports
• Dashboard setup & updates

31. Include Only The Important Stuff
1. No ‘vanity metrics’ – only the most important performance
metrics (KPI) with ability to drill down into details.
2. Context: current performance vs historical performance /
targets.
3. Smart use of visualizations – the right type and for the right
use case. “Making reports pretty” is not a waste of your time.

32. https://www.equest.com/wp-
content/uploads/2013/08/dashboard-snockered.png
Source: Equest

33. Avoid the ‘Curse of Knowledge’
Syndrome – Let Your Reports Talk
Include a short paragraph/bullet points of your main
observations:
1. Why is the performance the way it is?
2. What actions should be taken / what do you recommend to
improve the situation and drive business value?

34. Two Types of Output
BUSINESS ANALYSIS
• Hypothesis generation and
validation as a result of a
business question being
asked.
REPORTING
• Ad-hoc quantifications
• Quick data pulls
• Performance measurement
• Recurring reports
• Dashboard setup & updates

35. Business Questions Are…
1. Usually open-ended and at a much higher level,
leaving you room to think and add value.
Source: Avinash Kaushik
How can we increase
revenue by 5% in the
next three months?

36. Business Questions…
2. Likely require you to go outside your current systems and
sources to look for data and guidance in order to measure
success.
Source: Avinash Kaushik
What are the top 5
issues customers face
on our website?

37. Business Questions…
3. Rarely include columns and rows into which you can
plunk data you already have.
Source: Avinash Kaushik
What is the effect of
our website on our
offline sales?

38. Business Questions Warrant Hypotheses
No need for a PhD
Source: Wikimedia

39. The Hypothesis Framework (by Tim Wilson)
Source: Tim Wilson
I believe _________ (idea)
And if I am right, we will _________ (some action)

40. Example
What are the top 5
issues customers face
on our website?

41. Example I | E-Commerce
I believe our search result filters are not very intuitive
and don’t contribute to product findability.
And if I am right, we will need to revisit the filter structure and
introduce new logic in the filtering process.

42. Example II | E-Commerce
I believe people expect to be offered free delivery, which
we still don’t have.
And if I am right, we will have to consider the financial
implications of offering this service vs the current abandonment due
to shipping fees.

43. Forecast the Impact of Your
Recommendations (Monetization)
• Make you re-evaluate the usefulness of your proposals.
• Help you sell your ideas to senior stakeholders.
• Help the business prioritize the recommendations.
• Provide you with a proof of your value to the organization (also
keep track of post-release evaluations)

44. Monetization Framework (simplified)
Number of visitors on List Pages [monthly] 100,000
Interact with filters 30%
(CTR to Product Pages without filtering) 40%
(CTR to Product Pages after filtering) 25%
Expected increase in CTR after filter improvement [assumed] 15pp
Expected increase in traffic to Product Pages after redesign
[assuming filtering rate remains the same]
100,000 * 30% *15%
Conversion Product Pages to Cart 50%
Conversion Cart to Confirmation 60%
AOV 70 USD
Additional monthly revenue due to filter improvement 94,500 USD

45. To Sum It Up…

46. Making Your Analytics Talk Business I
Get your
stakeholders’
needs using
open
questions
Translate the
answers into
metrics, add
targets to derive
KPIs
Ensure you
collect the
right data to
measure those
KPIs

47. Making Your Analytics Talk Business II
Practice smart
performance
measurement
focusing on the
core KPIs
Spend more time
on hypothesis
validation based
on business
questions
Show the $$
impact of your
proposals on the
business

48. Thank you!
☺