These are slides from the ADPList mentoring session. https://app.adplist.org/dashboard/sessions/details/how-to-measure-and-communicate-the-impact-of-ux/about A timeless question and heavily debated one. I have been dumbfounded, defensive, even taken offense to this question in my career. With experience and willingness to revisit my opinions, I have learned that this complex question requires a considered response. Join this session if you want to explore the intent behind the question, the role of the product's maturity, the useful approaches to calculate the impact, the challenging circumstances, and craft a thoughtful point of view. By the end of this session, you should feel ready to discuss this question with empathy, confidence, and conviction.

1. How to measure and
communicate the value of ROI
Learn to refocus discussions about ROI

2. Unlock ROI and Impact PART 1

3. ROI ~ What do you need to know?
• The return on investment is a widely used financial metric for measuring the probability of
gaining a return from an investment
• It is a comparison metric useful in evaluating the potential return from a stand-alone
investment as it is in comparing returns from several investments

4. ROI ~ Disadvantages
 ROI does not account for how long an investment is held - the time value of money
 ROI does not account for the risk of the investment
NPV (Net Present Value) is an equivalent metric that offers the time value of money.

5. Calculation
ROI: Net Return on Investment / cost of investment ×100%
Net return: initial value - final value
Initial value: You invested $200 on an NFT.
Final value: You sold it for $250
Net return: $50
ROI: (50/ 200) * 100 = 25%

6. Reframe ROI as Impact

7. Impact
Means “having an effect or influence on someone or something”
Expands the boundaries of ROI to include perception and sentiment. It is VOI, value on
investment.
• Can help evaluate the potential worthiness of investments. But it does not necessarily
represent $ value.
• Can represent uptick in positive sentiment, customer advocacy, brand awareness.
• Can represent reduced risks to brand, earning customer trust.

8. Metric Savvy The cornerstone of
computing UX ROI

9. UX and Business Metrics
UX outcomes
How does delivering a great experience improve someone’s life?
What problems need to be solved to reduce friction for our users?
How can we add better customer value than our competitors?
Where can we delight our users to become a memorable brand?
Business Metrics
What should be true to increase our margin and revenue?
What needs to be true to reduce costs and save $?
How do we increase our differentiation among competitors?
How do we increase our market share?

10. UX and Business Metrics
UX outcomes
UX success Metrics
Indicate the progress towards making a true
difference in the user’s life
Business Outcomes
Key Performance indicators
Indicate the progress towards achieving the
business metrics

11. Business metrics & KPIs
Business Metrics KPIs
Increase in revenue Increased customers, profit margins, basket value,
margin value, donor growth and donations
Decrease in costs
Reduced customer support
Reduced cost of customer acquisition
Retention/churn Increased engagement (DAU)
Reduced # of customer exits
Satisfaction / Loyalty Increase in customer lifetime value (CLV), loyalty
(NPS) and referrals
Employee retention Increased productivity and turnover rate

12. UX Metrics and Methods
Qualitative
Perception
Usefulness
Perception of value
Intuitiveness
Autonomy
Empowerment
Perception of effort
Confidence to take action
Trustworthiness
Usability Metrics
( Qual + quant)
Success rate
Time on task
Error rate
Surveys
Satisfaction rating
Ease-of-use rating
Perceived usability
Questionnaire scores
(NPS, SUS, SUPR-Q)
Analytics
Time to adopt
# New accounts opened
Frequency of return visits
Feature usage
Conversion rate
Engagement rate
Renewal rate
Churn
Completion rate
Error counts & rate
Operational
# Customer-support tickets
# Phone calls, chats, emails,
texts
Training hours required
# Number of client
complaints
Average handle time
# First time resolutions

13. Dependencies Don’t forget when
measuring impact

14. Impact depends on 2 factors
1. The stage of the
product
2. The maturity of the
company

15. 1. The stage of the product
Early-stage products, until they establish market fit depend on qualitative metrics and
methods like generative UX research to:
• Reduce the risk of creating products that are not valuable to customers
• Unlock innovation opportunities that may not be evident but latent
Avoiding risks and opportunity costs are challenging to prove in terms of impact
Later stage product companies can and need to leverage qualitative metrics. If they only rely
on quantitative, that demonstrates their level of maturity.

16. 2: The maturity of the company
This plays a big role in how you can define and measure impact. Here are the signs:
• Low maturity companies depend on surveys and usability testing and basic UX design.
• Not a lot of focus to defining and tracking analytics to drive decision-making.
• Not a lot of focus to continuous discovery, design systems, interaction design, UX writing.
• Their decisions are mostly biased towards business metrics.

17. Connect the dots UX Metrics to Impact

18. In 4 steps
1. Understand the topline
and bottom-line business
metrics that matter
2. Understand the
associated KPIs and the
supporting calculation
3. Identify the UX metric
and the supporting
calculation
4. Validate the conversion
of UX metric and KPI
calculation with
stakeholders

19. Worked out example 1
1. Business Metric
Revenue from customer
subscriptions
2. KPI
Conversion rate
Customer NPS
3. UX Metric
Time to complete
onboarding tasks
Intuitive design to reduce
errors
4. Validate the
equation
Compare the cost of
design with the revenue
from conversion and the
impact of NPS

20. Worked out example 2
1. Business Metric
Save Marketing costs
and increase EBTIDA
(profitability)
2. KPI
Reduced CAC
(customer acquisition
cost)
3. UX Metric
Measure the no. of
referrals from the newly
designed referral page.
4. Validate the
equation
Compare the cost of design
with saved cost in
acquisition via referred
customers.

21. Worked out example 3
1. Business Metric
Market fit of a product
(MVP) by increasing
customer base and DAU
2. KPI
No. of customers willing
to sign up and use the
product
3. UX Metric
Users find the product
useful and usable.
Frequency and depth of
page visit
4. Validate the
equation
Compare the cost of
research and usability
testing to product
marketing, traffic, and
impact of bad reviews.

22. Reframe your perspective
Shh.. That has nothing to do with calculating ROI or assessing impact
PART 2

23. Misguided impulses

24. 1. Dumbfounded
Or awestruck.
• We presume that user experience is something everyone cares about.
• We assume that the premise of empathy and making it easier for users is just understood by
everyone.
Yes and…

25. 1.Dumbfounded Expect it!
As UXers the priority we assign to experiences need not be shared by stakeholders.
• Do we share the marketing manager’s concerns about choosing the right social channels to
show positive ROI?

26. Expect it. Initiate it
Let us:
• Expect it.
• Assume good intent that they are asking for understanding.
• Learn to communicate with calculation wherever we can.
• Influence to expand their view to include qualitative nature of the impact beyond numbers.
• As we get good at this, Initiate it!

27. 2. Getting offended
Alan Cooper once wrote that the value of design should be obvious to everyone in the
organization.
“If your boss is asking you to quantify the value of your work, you need to understand that your
work indeed has no value.
Not at that company. Not with that boss.”

28. 2. Getting offended
Alan goes on to suggest there are only two alternatives for this situation:
1) Accept that you and your situation are a valueless combination
2) Go some place where your work is valued. Go somewhere that doesn’t ask the value of your
work, but instead values your work!”

29. 2. Getting offended Be objective
Despite being Alan’s fan,
• This is an entitled point of view - Not everyone has the luxury to quit their jobs
• This is also misguided as it shuts down our role in being a change agent

30. 2. Getting offended Be objective
Don’t miss the signs.
• If you are an in-house UX professional, then some decision maker in your company has valued the
need for your role, your competency to hire you.
• That’s undeniable proof they see at least see some ROI / benefit / impact from UX.

31. 2. Getting offended Be objective
Let us acknowledge a few basic realities.
• A well managed business should allocate their resources thoughtfully to be viable.
• They should worry about where to spend the valuable dollars to retain customers.
• It is natural for teams to advocate for their own cause over another and a rational evidence-based
allocation is key.
• Our work is not academic – It is in service of enabling a business to make money by serving customers.
Not acknowledging these is not fair or a demonstration of empathy.
It shows a lack of awareness of the context within which our work shines.

32. 3. Being defensive
• We feel the need to defend design and educate our stakeholders.
• We start quoting examples and surveys to communicate the value of design.
This is not necessarily wrong or unhelpful. Sometimes misguided.

33. 3. Being defensive
When you are struggling to lose weight with a particular diet, think about the value of :
• One more motivational video
• A statistic from the health department on BMI
• A blog on cardio and weight training
What you will value is:
• Specific guidance on pitfalls of the diet you follow, you may not be aware of
• Helpful tweaks you can make to your diet

34. 3. Being defensive Be relatable
Providing examples from the industry is not going to help this business.
You want to find a way to connect the dots within your product, company and branding.
• Can you express the how the quality of the experience your design provides contributed to UX success
metrics (refer slide 10)
You need a baseline measurement to do that. Have you established that? that enables you to do that?
• Are you having cross team conversations? Can your customer success team endorse your efforts have
driven less number on onboarding related support calls? - More powerful than numbers)
Start there. Be relatable. Reshape the world view.

35. A reason to support ROI
and Impact conversations

36. Why measuring can be good?
How do you define or execute against a mission like:
“Create an outrageously seamless experience” “Create a super-amazing experience!”
How do you demonstrate that you have crafted a delightful experience to a peer team that
differs with you?
How do you ensure that your delightful experience is getting more delightful, adding more
value to the user?

37. Why measuring can be good?
Now consider this:
• Reduce abandonment rate by 5%
• Reduce support calls by 3% with better product page
• Increase conversion rate by 5% by simplifying onboarding
• Secure 70% or above in usefulness rating from customer interviews/ surveys (Early stage)

38. Why measuring can be good?
The definition of ROI or impact helps a team to rally around something
Inspiring,
Specific &
Tangible
that they can hold themselves accountable to.

39. The next time… When people ask about ROI,
don’t answer it immediately.

40. You know to be better
• You will not be dumfounded and initiate the conversation.
• You know to empathize without getting offensive or defensive.
• You know to define, create and measure impact accounting for the stage of the product
and the maturity of the organization.

41. You will explore deeper
You will know to gather:
• what the company cares about – what they openly tout not what they just claim.
• what part of the product worries them.
• what about the customers they want to learn about.
• what are the known problems about measuring.
• what they are being measured on.
• what you are being measured on.

42. You know to work the 4 steps
1. Understand the topline
and bottom-line business
metrics that matter
2. Understand the
associated KPIs and the
supporting calculation
3. Identify the UX metric
and the supporting
calculation
4. Validate the conversion
of UX metric and KPI
calculation with
stakeholders

43. You will feel ready to
To discuss this topic with
empathy, confidence, and conviction
and lead with a thoughtful point of view!

44. Appendix
Learn to refocus discussions about ROI

45. Hi . I work in the realm of
Experience, growth and strategy,
the 3 pillars of innovation.
I am also the founder of
careernext.me
This is me at LinkedIn.
Appreciate you sending a message with your invite.

46. Links and
References
ROI foundations
Detailed examples on ROI calculation
Data driven vs Data informed
Calculating ROI
Setting Metrics
Challenges of identifying UX success metrics
Opportunities with ROI conversations
Metrics vs Experience
3 Myths about ROI

47. Metric Nuances Without which there’s
no savvy

48. Qualitative metrics cannot tell you
• The degree to which people love, hate, or are indifferent to your product or any of its specific features
• Whether a change increases or decreases people’s trust in your product over time
• How simple and easy to use your product is perceived to be
• How people see your product versus other similar products in the market
• What things people most want changed, added, or fixed
• How people will want to use your product as time passes

49. Quantitative metrics cannot always
• Equip you with the adequate confidence to make decisions
• Exception: PRE product-market fit phase.
• Reduce the likelihood of risks – but they cannot guarantee success. Actual user behaviors do.

50. You need the right triangulation of
of metrics
• Qualitative, Quantitative and competitive
• Lagging and Leading metrics
• Outcome and Progress metrics
To make data-informed decisions – beyond data driven and metric savvy.

51. ROI ~ IRR and NPV
ROI vs IRR (Internal rate of return)
ROI indicates total growth, start to finish, of an investment, while IRR identifies the annual
growth rate. While the two numbers will be roughly the same over the course of one year, they
will not be the same for longer periods.
ROI vs NPV (Net Present Value)
NPV measures the cash flow of an investment; ROI measures the efficiency of an investment. ...
NPV calculates future cash flow; ROI simply calculates the return that the investment produces.
NPV cannot determine the dedicated investment; ROI can be easily manipulated to the point of
inaccuracy

52. Impact depends on 2 factors
1. The maturity of the company plays a big role in how you can define and measure impact.
2. The stage of the product plays a critical role in how you can define and measure impact.