You'll learn: - How to create a UX ROI model with decision trees and expected values - How to forecast the effect of UX on sales - How to use SUS and NPS to measure the effect of UX

1. How to predict the $
value of UX using
standard financial
models
By Aaron Powers

2. Presentation & Research Supported By

3. Goal: Teach techniques to predict $
value of UX before doing the UX work.

4. What can UX improve about a business?
Increase Sales
Reduced tech support calls
Customer productivity & happiness

5. UX “professionals are called upon to take businesses with an
interest in reaching broader audiences and make them
accessible to every segment of the market”
–- Eugenie Bertus & Mark Bertus, “Determining the Value of
Human Factors in Web Design”

6. How can UX improve sales growth?
Learnability
Transfer to other products
Recommendations to others
Renewals

7. Return On Investment (ROI)
7
The result could be: “ROI of UX is 10%”

8. Return On Investment (ROI)
Challenges to using the ROI calculation in UX:
You must be able to financially separate:
The gain from UX
The gain from all other efforts (e.g. development of new
features)
Estimates can be perceived as biased by the person creating
them.
8

9. ROI & More
9

10. Method 1: Estimate the value
of individual UX decisions with
a decision tree
Method 2: Model how
investments influence
business results

11. How can you use these?
• Use these models directly, in
full or in part
• Use as a thinking framework
OR

12. Concepts Drawn From Diverse Fields
Finance
Textbook
financial
modelling
including pro
formas
13
Decision
Models
Textbook
models for
simulating &
guiding
decision
making
Marketing
Analytics
Marketing
analytics has a
lot of common
goals in
connecting user
behavior to
business results
Machine
Learning
Regression
modelling can
be used to cut
through noise
and build
models
between user
behavior &
business results.

13. Method 1: Decision Tree
Example: Early User Research

15. Decision Point

16. Cost

17. Final Value

18. Cost

19. Probability

20. Revenue

21. Final Value

22. The expected value … is the weighted average of all values…
The value may not be expected in the ordinary sense—the
"expected value" itself may be unlikely or even impossible (such
as having 2.5 children)…

24. Expected Value (weighted average)
Weighted average of market
responses:
.45 * 4m +
.35 * -1m +
.20 * -6m
= Expected Value = 250k

25. Expected Value

26. Simulation Results

29. What happens if we include user
research up front to understand risk
better?

30. Benefit #1 of user research
Better estimation on whether the market
response will be
Great:
Awful:
Fair:
Possible test results of a study with 5 people:

31. Great
Fair
Awful
If test result is: Market response probability
becomes:

32. Add a decision here
whether to run a study first

36. Benefit #2 of user research
Early ability to tune overall product design so
that you’re more likely to have a great
market response

39. So, how much should we spend on
running this kind of study?

40. EVSI=Expected Value of Sample
Information
Value of sample information =
Expected Value of Doing Test
(if test were free)
– Expected Value Without Test
= Pay up to $987k for UX benefit #1
or up to $1.25m for both UX benefits

41. Method 2: Model how investments
influence business results
Example: Model The Value of
Organization Wide Investments In UX

42. Based on work by others
44
Len Schlesinger
Harvard Business School
Professor
Jeff Sauro
Author of Quantifying the
User Experience
Jim Lewis
IBM Master Inventor with
77 patents

43. Three Stages
Model which factors influence
revenue for your company
Technique: Regression modelling & mediation analysis
Model influencers
Technique: Measure correlations from investment
through to results
Calculate or predict the impact
Technique: Financial Pro formas

44. The Service Profit Chain

45. The Service Profit Chain

46. Model Factors In Between
48
Better
teachers
Higher
graduation
rates

47. Model Factors In Between
Better
teachers
Better
grades
49
Higher
graduation
rates

48. Model the layers in between the business goal & UX
51

49. Step 1: Model the effect of UX

50. Model How UX Affects the Business
Improved User
Experience
Better
perception of
product
Better Word Of
Mouth
Sales Growth
Increased
Productivity
Awareness of
savings
compared to
alternatives
Higher renewal
rates
Lower Barriers
To Entry
More Users
able to use
product
Sales Growth

51. How
Improved User
Experience
Better
perception of
product
Better Word Of
Mouth
Sales Growth
The most common model is:
Each has popular metrics:
The best part is that we’re not doing anything new – every step
has been done before.
Investment In
User
Experience
Higher System
Usability Scale
Higher Net
Promoter
Score
Sales Growth

52. How
Improved User
Experience
Better
perception of
product
Better Word Of
Mouth
Sales Growth
The most common model is:
Each has popular metrics:
The best part is that we’re not doing anything new – every step
has been done before.
Investment In
User
Experience
Higher System
Usability Scale
Higher Net
Promoter
Score
Sales Growth

53. Investment In
User
Experience
Higher System
Usability Scale
Higher Net
Promoter
Score
Sales Growth
How

54. Measure Usability (with SUS)

55. Measure Usability (with SUS)

56. Investment In User
Experience
Higher System
Usability Scale
Higher Net
Promoter Score
Sales Growth
How

57. Investment In User
Experience
Higher System
Usability Scale
Higher Net
Promoter Score
Sales Growth
How

58. Net Promoter Score

59. Collect NPS or estimate from SUS

60. Investment In User
Experience
Higher System
Usability Scale
Higher Net
Promoter Score
Sales Growth
How

61. Investment In User
Experience
Higher System
Usability Scale
Higher Net
Promoter Score
Sales Growth
How

62. NPS is correlated with Sales Growth

63. Step 2: Get some financial data

64. Get a few numbers from the friendly guy in
the finance department
Current revenue
Number of years to simulate
Annual growth rate
Tax rate
EBIT margin
Capital Expenditures as a % of
revenue
Working capital as a % of
revenue
Depreciation as a % of
revenue
Weighted average cost of
capital
Cost of equity
Cost of debt
Weight of equity
Weight of debt

65. Forecast The Next Few Years

66. Example: An Average Software Company

67. Example: An Average Software Company

68. Investment In User
Experience
Higher System
Usability Scale
Higher Net
Promoter Score
Sales Growth
How
Software Industry Averages
68 -15% 14%
Target
80 20% 19%
3% of spending
9% of spending

69. Example: An Average Software Company
Increasing SUS from average to best has the potential to increase
sales growth 3.1%-8.4% per year
The financial returns on average would be:
A 67% internal rate of return (IRR)
54% Return on Investment (ROI)

70. Method 1: Model the value of
individual UX decisions with a
decision tree
Method 2: Model the value of
organization wide investment
with pro formas

71. How to predict the ROI
of UX using standard
financial models
By Aaron Powers
apowers@athenahealth.com
linkedin.com/in/aaronpowers